WordPress Importer Redux

Hi, I’m Ryan McCue. You may remember me from such projects as the REST API.

I’m here today to talk about something a bit different: the WordPress Importer. The WordPress Importer is key to a tonne of different workflows, and is one of the most used plugins on the repo.

Unfortunately, the Importer is also a bit unloved. After getting immensely frustrated at the Importer, I figured it was probably time we throw some attention at it. I’ve been working on fixing this with a new and improved Importer!

If you’re interested in checking out this new version of the Importer, grab it from GitHub. It’s still some way from release, but the vast majority of functionality is already in place. The plan is to eventually replace the existing Importer with this new version.

The key to these Importer improvements is rewriting the core processing, taking experience with the current Importer and building to fix those specific problems. This means fixing and improving a whole raft of problems:

  • Way less memory usage: Testing shows memory usage to import a 41MB WXR file is down from 132MB to 19MB (less than half the actual file size!). This means no more splitting files just to get them to import!
  • Faster parser: By using a streaming XML parser, we process data as we go, which is much more scalable than the current approach. Content can begin being imported as soon as the file is read, rather than waiting for pre-processing.
  • Resumable parsing: By storing more in the database instead of variables, we can quit and resume imports on-the-go.
  • Partial imports: Rethinking the deduplication approach allows better partial imports, such as when you’re updating a production site from staging.
  • Better CLI: Treating the CLI as a first-class citizen means a better experience for those doing imports on a daily basis, and better code quality and reusability.

Curious as to how all of this is done? Read on!

Continue reading


REST API: Welcome the Infrastructure to Core

Hi from the REST API team! We’re extremely excited to announce the API infrastructure has now been merged into core as of r34928 (plus a couple of fix up commits we won’t mention). Huzzah!

Sincere thanks to every single one of the contributors, we wouldn’t be where we are today without you. It takes time and effort to produce great things, and it’s impossible to make things great without everyone helping. This has been a truly collaborative effort, and I wish I could do more than just give you props.

(Important note: if you have a 2.0 beta already installed, you must upgrade to beta 5.)

What’s included in 4.4?

As mentioned in the merge proposal, the API comes in two parts: infrastructure and endpoints. In 4.4, the infrastructure is now available as part of core, while the endpoints continue to only be available in the plugin.

You can think of the infrastructure as an “API construction kit”. WordPress 4.4 will make it possible for everyone to build RESTful APIs in a much easier fashion, which will benefit people building custom APIs for their site. The infrastructure handles the routing, argument handling, JSON serialisation/deserialisation, status codes, and all that other lovely REST stuff.

For client authors, this doesn’t help you much right now. Hold tight, the team is working as fast as we can on the endpoints to get them ready for a future release. In the meantime, you can make sure sites you want to work with have the plugin installed, which isn’t a change from the current state.

For plugin and theme authors, you can start building new APIs immediately using the infrastructure now in core. This can start replacing your existing custom admin-ajax endpoints or other bespoke code you already have.

To authenticate with the API, only built-in cookie authentication is available out of the box right now. The OAuth 1 plugin will continue to work with WP 4.4 and the API plugin, as will the Basic Auth plugin for local development.

It’s super easy to get started, and there’s even a guide available to kick-off. (Note: the WP_REST_Controller class is not included in WordPress 4.4.) This documentation will be migrated across to developer.wordpress.org soon.

If you want to access any of the built-in data in WordPress without building it out yourself, you’ll need the endpoints as well. These will continue to be packaged in plugin form, and version 2.0 final will be released to accompany 4.4 before the end of this cycle.

What if I’m using the API already?

If you’re on version 2 of the API, you’ll need to update the API to beta 5 or later before updating to the latest version of core. This new version will use the new infrastructure in core if available, or fallback to a compatibility library otherwise.

Important note: Earlier 2.0 betas (1 through 4) are incompatible with WP 4.4. Your site will fatal error if you don’t upgrade to beta 5 or later. You must upgrade to the latest API to run trunk and to run WP 4.4 when it’s released.

If you’re on version 1 of the API, you won’t hit any fatal errors straight away, but endpoints will stop working with 4.4. We’re still planning on releasing a final version 1 release for compatibility, but now would be a great time to consider migrating forward to version 2. Apart from security releases, version 1 has ceased being actively maintained.

Looking forward for the API

Now that the API is past this first hurdle, it’s important to keep looking forward. Our immediate next step is to improve and polish the endpoints for phase two of our merge. There’s a lot of work still to be done here, so we’d love you to join us on GitHub.

The infrastructure of the API will now be maintained via Trac, so new issues and patches should be sent there instead under the “REST API” component. Issues with endpoints should still be filed on GitHub. Don’t worry if you’re not sure; you can file issues on either Trac or GitHub, and they’ll be triaged into the correct place as needed. (It’s more important to make sure the issue is filed in the first place!)

The team wants to keep pressing forward with the API and keep up our rate of progress, if not improve it even further, and we’d love your help. We still need help from content writers on our documentation, designers and developers on our authentication plugins, and developers on the endpoints. If you want to help, we can always use a hand, and we’d love to help get you started. We’re available in the #core-restapi room on Slack, and we’d love to see you at the weekly meeting at Monday 23:00 UTC 2015.

We look forward to continuing to work on the API and getting these endpoints happening. Thanks again to everyone who got us here.

(Then again, maybe a REST API is more of a Shelbyville idea…)

#4-4, #dev-notes, #json-api, #rest-api

WP REST API: Merge Proposal

Hello everyone! This is the post you’ve all been waiting for. 🙂

We on the REST API team (myself, @rachelbaker, @joehoyle, @danielbachhuber, and newest member and core committer @pento) would like to propose merging the REST API into WordPress core. We’ve been working a while on this, and think it’s now ready to get your feedback.

This is our first iteration of the proposal, and we’re actively looking for feedback. If you have thoughts on the project, or on this proposal, let us know! Only with your feedback can we make progress. 🙂

What is the REST API?

The REST API is a nice and easy way to get at your data in WordPress externally, whether that’s from JavaScript in a theme or plugin, mobile and desktop applications, or importing and exporting data. The API offers up all core data types (posts, terms comments, and users), plus support for meta and revisions; we’ve got plans to eventually have access to everything the admin and frontend have access to.

The REST API differs from existing WordPress APIs in that it is explicitly designed from the ground up for modern mobile and browser usage, using the lightweight and widely-supported JSON data serialization format with a modern REST interface. Both of these are already familiar to most developers: JSON is a subset of JavaScript intended purely as a data interchange format, and REST is a set of best practices around HTTP. Both are supported natively by almost every programming language and platform.

Why do we need a new API?

WordPress already has external APIs: XML-RPC, designed for desktop clients; Atom and RSS feeds, designed for post syndication; and the venerable admin-ajax, designed for Ajax requests in the admin and frontend. These APIs all serve different purposes, but often have a great deal of overlap. In addition, these have all been stretched beyond their original intentions: XML-RPC now contains site management tools, RSS has been extended into the WXR export format, and admin-ajax is the catch-all of any sort of browser-server communication in plugins and themes.

The REST API builds upon the heritage of these APIs to provide better support today for using these, as well as laying the groundwork for expanded use in the future.

XML-RPC is the closest analogue to the REST API in terms of usage and capabilities. Originally designed back in 1998 to allow desktop clients to create and edit posts on blogs, WordPress has extended this with both other specifications (such as MetaWeblog) and with its own proprietary additions. Fundamentally, XML-RPC is built around Remote Procedure Calls (RPC), essentially a way of calling a function externally. It then uses XML to serialize the data for passing back and forth.

Unfortunately, XML serialization can be problematic at times. XML has lots of power, but support for custom entities and DOCTYPEs can cause parsing problems and security attacks, including billion laughs, and XXE exploits. (Currently, WordPress has to disable parts of the XML parser and apply regular expression replacements to be able to parse these safely.) XML is also very verbose, and represents data in a way which doesn’t map easily to programmatic data structures. JSON on the other hand is both concise and well-represented in memory, as it’s based on JavaScript’s native syntax.

The admin-ajax API is also very commonly used in WordPress, albeit typically only by plugins and themes. This is a very lightweight API that essentially acts as a mini-router. Typical usage of this API uses JSON, since it’s a browser-based API, but all usage is completely custom. A lot of the usage of this involves retrieving or updating posts on-the-fly, but due to its nature as simply a framework, these are done in completely different ways. This doesn’t lead itself to extensibility, and requires a lot of duplication every time developers want to get data in or out. We don’t want to replace all of admin-ajax though, since some use cases don’t map exactly: UI-related interactions or things like the Heartbeat API are great examples of this.

The REST API can help to supplant these uses. Our aim is to eventually replace the XML-RPC API completely, to act as a secondary import/export format, and to replace most (but not all) uses of admin-ajax. The REST API offers an easier to use interface than the existing solutions.

Why this project?

We’ve been working on this project ever since the first WordPress Contributor Summit back in 2012. Since then, we’ve had lots of feedback from core developers, the community at large, and further beyond in the form of client developers. We believe that the REST API has an immense amount of experience behind it, and plenty of viewpoints have contributed to the project’s direction.

The API has seen great usage in the community, from various mobile apps to large news sites. This usage has helped to battle-test and prove out the API. In the process, we’ve found plenty of bugs, and some security issues. Thanks to this feedback, the API is incredibly stable and secure. (The most recent security bugs we fixed were relatively minor bugs.)

We also designed the API from the ground-up to be part of core, following a core-like mentality to our processes. The API is intended to be both a great feature and a base to build off in plugins. We undertook a significant refactoring and partial rewrite in version 2 to make this extensibility even better. This open process also means that most of the design decisions are documented publicly as we’ve engaged stakeholders to gauge feedback.

We believe these pieces combined make this a fantastic feature for WordPress core, and we hope you all agree. 🙂

What’s the plan?

The plan we’re aiming for is a two part merge of the API. For the first stage, the infrastructure code would be merged into wp-includes and made available for plugins and themes. This is an internal API only, but offers an “API construction kit” for developers to use. For the second stage, the endpoints would be merged, and the API would be enabled for sites by default.

This plan splits the API into two parts, infrastructure and endpoints:

  • Stage One: Infrastructure: The infrastructure is the code responsible for routing requests and handling the “meta” layer of the API, including JSON serialisation/deserialisation, linking, embedding, and REST best practices. This adds a simplified routing layer outside of WP’s rewrites, allowing non-query-var rewrites easily, acting as a base for building APIs inside WordPress.
  • Stage Two: Endpoints: These are where much of the complexity of the API lies, as they’re responsible for mapping data from the external JSON format to the internal data structures, and vice versa. The “business” logic of integrating with WordPress is almost entirely contained within the endpoints. These are the more complex part of the API, as they require using deep APIs in WordPress, and handling security and privacy concerns.

With this plan, we would aim to have the infrastructure merged in 4.4, and the endpoints merged one release later in 4.5.

The slow nature of this plan allows a longer review time on the API for core committers. It also gives extra time for reviewing the endpoints, since they would be delayed one release.

Merging the infrastructure now would allow third-party code to begin using the API to build upon it, including migrating from existing custom code. It would also help to increase developer confidence in the API (as it represents a commitment by the project towards a REST API).

In this plan, the first stage would not include any of the base controllers (such as the posts controller). This may limit the utility of the infrastructure for plugins and themes, however as the endpoints would be merged a cycle later, it’s expected that this wouldn’t have a huge impact.

The infrastructure of the API is approximately 2700 lines of code (roughly a third of the API codebase), and the endpoints make up the remaining 5500 lines of code.

What would happen after merge?

After merging the REST API into core, we’d spend approximately two weeks partying and celebrating. 🙂

Once we’re done with the parties, one major question would be how we manage the API in the future. The existing management and contribution process via GitHub has been extremely successful, as we’ve had 61 people’s pull requests merged into the API. Contribution via GitHub is especially useful for the API, as it’s a heavily developer-focussed project, and is aimed at external (non-WordPress) developers. On the other hand, all other contribution to WordPress is done via SVN and Trac, so integrating with this process is important for existing developers, as well as core’s general processes. We need to ensure the API is an integral part of core, not a separate project.

Given the team’s experience with GitHub as well as Trac, we can bring the best of both worlds by helping integrate the two. This would also improve contribution for WordPress as a whole, and benefit the whole community. This will be especially important as we encourage more contributions from the wider community (client authors, for example). We think we can make good progress here, and we’d love to try and help improve the process. 🙂

In addition to the project management, there are still further API projects we need to tackle. Authentication is the most important of these, as a huge focus on OAuth and similar would be needed to make the API more useful for external clients. Currently, we haven’t had enough time to spend on this as well as managing the API project, however the API is now reaching a finalised stage, so this should be able to improve quickly. Centralised authentication is a huge part of this, as the regular OAuth registration process is cumbersome for a distributed system like WordPress.

Important note: We don’t believe authentication is required for the API merge, and we treat it as a separate project. The authentication is a completely separate system to the API itself. This is something we’d give more time to in the future, but we want to focus on shipping the API for now.

Let’s go!

This is our merge plan for the API, however it’s not finalised. If you’ve got comments, thoughts, suggestions, opinions, or words of encouragement, let us know. We’d love to know what you think. Thank you all, you’re wonderful, and stay golden.

#feature-plugins, #json-api, #merge, #proposal, #rest-api

WP REST API: Versions 1.2.3 (Security Release) and 2.0 Beta 4

First and foremost: version 1.2.3 of the REST API is now available. Download it from the plugin repository or from GitHub. This is a security release affecting sites running version 1.2 or a 2.0 beta releases.

Security Release

Recently, we were alerted to a potential XSS vulnerability introduced in version 1.2 of the API related to the JSONP support. This vulnerability also existed in version 2.0. Thanks to Alex Concha (@xknown) for reporting this issue to the team responsibly.

This release was coordinated by the REST API team and the WordPress core security team. The security team is pushing automatic updates for version 1.2.3, but do not wait or rely on the automatic update process. We recommend sites or plugins that are using either v1.2.x or 2.0 beta releases update the plugin immediately.

If you’d prefer not to upgrade, you can instead disable JSONP support through a filter. For version 1:

add_filter( 'json_jsonp_enabled', '__return_false' );

To disable JSONP on version 2:

add_filter( 'rest_jsonp_enabled', '__return_false' );

If you have a question about the security release, you can find the team in #core-restapi on WordPress.org Slack, or you can privately message @rachelbaker, @rmccue, @danielbachhuber, or @joehoyle.

Version 2.0 Beta 4

Alongside the security release for version 1.2, we’re also releasing the latest beta for version 2.0: 2.0 Beta 4 “See My Vest”. You can download this from the plugin repository or from GitHub.

This beta release includes the security fix from version 1.2.3, so we recommend everyone running a version 2 beta update immediately to fix the issue.

As well as the security release, this beta also includes a bunch of other changes. Here’s some highlights:

  • Show public user information through the user controller.

    In WordPress as of r32683 (scheduled for 4.3), WP_User_Query now has support for getting users with published posts. To match current behaviour in WordPress themes and feeds, we now expose this public user information. This includes the avatar, description, user ID, custom URL, display name, and URL, for users who have published at least one post on the site. This information is available to all clients; other fields and data for all users are still only available when authenticated.

  • Send schema in OPTIONS requests and index.

    Rather than using separate /schema endpoints, the schema for items is now available through an OPTIONS request to the route. This means that full documentation is now available for endpoints through an OPTIONS request; this includes available methods, what data you can pass to the endpoint, and the data you’ll get back.

    ⚠️ This breaks backwards compatibility for clients relying on schemas being at their own routes. These clients should instead send OPTIONS requests.

  • Update JavaScript API for version 2.

    Our fantastic JavaScript API from version 1 is now available for version 2, refreshed with the latest and greatest changes. Thanks to Taylor Lovett (@tlovett1), K. Adam White (@kadamwhite) and Nathan Rice (@nathanrice).

  • Embed links inside items in a collection.

    Previously when fetching a collection of items, you only received the items themselves. No longer! You can now request a collection with embeds enabled (try /wp/v2/posts?_embed).

  • Move /posts WP_Query vars back to filter param.

    In version 1, we had internal WP_Query vars available via filter (e.g. filter[s]=search+term). For our first betas of version 2, we tried something different and exposed these directly on the endpoint. The experiment has now concluded; we didn’t like this that much, so filter is back.

    ⚠️ This breaks backwards compatibility for users using WP Query vars. Simply change your x=y parameter to filter[x]=y.

  • Respect rest_base for taxonomies.

    ⚠️ This breaks backwards compatibility by changing the /wp/v2/posts/{id}/terms/post_tag endpoint to /wp/v2/posts/{id}/tag.

As always, we have a detailed changelog as well as the full set of changes if you’re interested.

(Note that while this version 2 beta breaks backwards compatibility, the 1.2.3 security release does not break compatibility with the 1.2 branch.)

This release had 11 contributors, and we’d like to thank each and every one of them:

$ git shortlog 2.0-beta3...2.0-beta4 --summary
     1   Daniel Bachhuber
    11   Daniel Jalkut
     1   Fredrik Forsmo
     1   Jared Cobb
     3   Jay Dolan
    26   Joe Hoyle
    10   Josh Pollock
    25   Rachel Baker
    50   Ryan McCue
    24   Stephen Edgar
     8   Taylor Lovett

Thank you again to all of our beta testers, and thanks to everyone who let us know how you’re using the API. We’re taking note of all of your feedback, and you might see some further changes related to that in coming releases.

#feature-plugins, #json-api, #rest-api, #security, #updates

REST API: Who’s Using This Thing?

Hi everyone!

This is a break from your regularly scheduled release posts. We’re looking to gather some feedback on the lead up to merging into core, to assess what your thoughts are on the API. Whether you’ve used the API or not, we’d love to hear your thoughts.

Here’s a bunch of questions to start you off:

  • What are you doing with the API?
  • What would you like to do with it? (How can we make it better for you? What can’t you do right now that you need?)
  • What improvements would you like to see? (What sucks?)
  • Is the API important to you?
  • Anything else you’d like to tell us?

We really want to make sure this API fits your needs. Without your support, the API really means nothing, so we want to make sure we get this basically perfect first. We’d love to hear feedback from everyone using this, from JS-only developers coming to WP for the first time, through WordPress plugin and theme developers, all the way through to PHP developers not involved with WordPress.

If your comment is too long to fit here, or you’d like to wax lyrical about the API, feel free to comment on your own blog and cross-post a link across to here.

If you can’t comment publicly due to disclosure, you can leave a message for me personally at me at ryanmccue.info. Please specify if you will allow me to share your private feedback with the WP REST API project and core team, or if you’d prefer to keep it completely private between the two of us.

Help us make this the best feature to ever land in WordPress. 🙂

#json-api, #rest-api

WP REST API: Version 2.0 Beta 1

Hi there, we’re the REST API team. You may remember us from such releases as Version 1.1 “Hammock Hut” and Version 1.2 “S-M-R-T”. Today, we’re excited to announce the first beta release of version 2 of the REST API: 2.0 Beta 1 “Ralph Wiggum”. Go and grab it from GitHub right now, we’ll wait.

Over the past 9 months, we’ve been reworking some of the important internals of the REST API to improve extensibility for site developers, and usability for both plugin developers and client developers. We’ve codified some of our important internal conventions, added plenty of new tools for you to play with and simplify your development, and reworked some of the less fantastic parts of the API. We think these changes will make the API a whole lot better for everyone, and we’re excited to start getting your feedback on these changes.

One of the things that you’ll notice immediately when using version 2 is just how much hasn’t changed. There’s a lot of things that we’ve changed under the hood, but the fundamentals of the API haven’t changed up too much. Version 2 is a direct continuation of version 1, so most of the API will be instantly familiar to you if you’ve used version 1 before. We have a primer on the important changes since version 1, including a guide on migrating existing endpoints across to version 2.

Here’s some super important notes on version 2, before you get started:

  • No Backwards Compatibility: Version 2 is not backwards compatible with version 1. We have plans to ensure that a version 1 compatibility layer exists, but we’ve leaving that until the API has settled down fully before creating that. However, porting endpoints is easy if you want to give it a go.
  • No Forwards Compatibility: Beta 1 is also not guaranteed to be forwards compatible with future betas. Although this is a beta release, it’s still likely that parts of the REST API will change. However, at this point, the core infrastructure of the API is unlikely to change in a breaking manner, so internal plugin APIs are pretty solid. We reserve the right to break these in future though, so keep that in mind when updating.
  • Development Only: Due to the lack of forwards and backwards compatibility, we strongly recommend you do not run the beta in production environments. While it is possible to run it (both security and privacy have solid, thorough handling), any updates will likely break your site.
  • Run Concurrently: You can easily run both version 1 and 2 side-by-side, since the prefixes for all functions and filters have changed. This will require changing the URL for version 2 however, which can be done with the following filter:
    add_filter( 'rest_url_prefix', function () { return 'wp-rest'; } );

With all this said, we’d love you to test the plugin as soon as possible. We’re looking for feedback on every part of the API, especially the changes since version 1. If there’s anything that has made it easier or harder since version 1, let us know. We’re continuing to improve the API every day, and now is the time to let us know what’s important to you. (P.S. Did I mention we have an issue tracker?)

We’re excited for this new stage of the API, and we hope you are too.

#json-api, #rest-api

JSON REST API: Version 1.1.1 (Security Release)

I’d like to announce the availability of version 1.1.1 of the JSON REST API. This is a security release for a minor security issue, however we recommend all users running 1.1 upgrade as soon as possible.

This release only affects users running WP API on a domain with other (non-WordPress) software running. Using the JSONP support built-in to the API, it is possible to serve up arbitrary Flash SWF files from the API, allowing these Flash files to bypass browser cross-origin domain policies. While WordPress includes built-in CSRF protection, other software running on the same domain may not include similar protections.

As a workaround, JSONP support can be disabled on your site with:

add_filter( 'json_jsonp_enabled', '__return_false' );

Thanks to @iandunn for reporting this issue to the team responsibly.

We’d also like to announce that WP-API is now available on HackerOne. We invite security researchers and developers to report any potential security issues to us via HackerOne, allowing us to triage and fix issues privately, and also award bounties for valid security reports.

#json-api, #rest-api

JSON REST API: Version 1.1

I’m happy to announce the availability of version 1.1 of the JSON REST API.

This release is a bit of a smaller, more focussed release as we work on increasing test coverage and squashing bugs. Here’s the juicy details:

  • Add new routes for taxonomies and terms.

    Taxonomies and terms have now been moved from the /posts/types/<type>
    namespace to global routes: /taxonomies, /taxonomies/<tax>,
    /taxonomies/<tax>/terms and /taxonomies/<tax>/terms/<term>

    Test coverage for taxonomy endpoints has also been increased to 100%.

    Deprecation warning: The /posts/types/<type>/taxonomies endpoint (and
    sub-endpoints with the same prefix) have been deprecated in favour of the new
    endpoints. These deprecated endpoints will now return a
    X-WP-DeprecatedFunction header indicating that the endpoint should not be
    used for new development, but will continue to work in the future.

    (props @kadamwhite, @rachelbaker, @rmccue, #198, #211)

  • Allow customizing the API resources prefix

    The API base (typically wp-json/) can now be customized to a different
    prefix using the json_url_prefix filter. Note that rewrites will need to be
    flushed manually after changing this.

    (props @ericandrewlewis, @rmccue, #104, #244, #278)

  • Give null as date for draft posts.

    Draft posts would previously return “0000-00-00 00:00:00” or
    “1970-01-01T00:00:00”, as draft posts are not assigned a publish date. The API
    now returns null where a date is not available.

    Compatibility warning: Clients should be prepared to accept null as a
    value for date/time fields, and treat it as if no value is set.

    (props @rmccue, #229, #230)

  • Fix errors with excerpt.

    Posts without excerpts could previously return nonsense strings, excerpts from
    other posts, or cause internal PHP errors. Posts without excerpts will now
    always return an excerpt, typically automatically generated from the post

    The excerpt_raw field was added to the edit context on posts. This field
    contains the raw excerpt data saved for the post, including empty
    string values.

    (props @rmccue, #222, #226)

  • Only expose email for edit context.

    User email addresses are now only exposed for context=edit, which requires
    the edit_users permission (not required for the current user).

    The email address field will now return false instead of a string if the
    field is not exposed.

    (props @pkevan, @rmccue, #290, #296)

  • Correct password-protected post handling.

    Password-protected posts could previously be exposed to all users, however
    could also have broken behaviour with excerpts. Password-protected posts are
    now hidden to unauthenticated users, while content and excerpts are shown
    correctly for the edit context.

    (Note that hiding password-protected posts is intended to be a temporary
    measure, and will likely change in the future.)

    (props @rmccue, #286, #313)

  • Add documentation on authentication methods.

    Full documentation on authentication
    is now available. This documentation explains the difference between the
    various available authentication methods, and notes which should be used.

    (props @rmccue, #242)

  • Include new client JS from github.io

    The WP-API Javascript library is now loaded dynamically from
    wp-api.github.io to ensure it is always up-to-date.

    (props @tlovett1, #179, #240)

  • Don’t allow setting the modification date on post creation/update.

    As it turns out, WP core doesn’t allow us to set this, so this was previously
    a no-op anyway. Discovered during test coverage phase.

    (props @rachelbaker, @rmccue, #285, #288)

  • Check post parent correctly on insertion.

    Posts could previously be added with an invalid parent ID. These IDs are now
    checked to ensure the post exists.

    (props @rmccue, #228, #231)

  • Make sure the type is actually evaluated for json_prepare_${type} filter.

    This value was previously not interpolated correctly, due to the use of the
    single-quoted string type.

    (props @danielbachhuber, #266)

  • Return WP_Error instead of array of empty objects for a revisions
    permissions error.

    Previously, when trying to access post revisions without correct permissions,
    a JSON list of internal error objects would be returned. This has been
    corrected to return a standard API error instead.

    (props @rachelbaker, @tlovett1, #251, #276)

  • Flip user parameters check for insert/update.

    Previously, you could add a user without specifying username/password/email,
    but couldn’t update a user without those parameters. The logic has been
    inverted here instead.

    (props @rmccue, #221, #289)

  • Add revision endpoints tests

    (props @danielbachhuber, @rachelbaker, @rmccue, #275, #277, #284, #279)

  • Add post endpoint testing

    Now at >54% coverage for the whole class, and >80% for the main methods. This
    figure will continue to rise over the next few releases.

    (props @rachelbaker, @rmccue, #99)

  • Separate helper functions into global namespace.

    WP_JSON_Server::get_timezone(), WP_JSON_Server::get_date_with_gmt(),
    WP_JSON_Server::get_avatar_url() and `WP_JSON_Server::parse_date() have
    all been moved into the global namespace to decouple them from the server

    Deprecation warning: These methods have been deprecated. The new
    json_get_timezone(), json_get_date_with_gmt(), json_get_avatar_url() and
    json_parse_date() methods should now be used instead.

    (props @rmccue, #185, #298)

As always, we’ve got a full list of all the changes and a longer changelog. Here’s who contributed to this release:

$ git shortlog 1.0...1.1 --summary
     8  Daniel Bachhuber
    12  DrewAPicture
     3  Eric Lewis
     2  JDGrimes
     9  K.Adam White
    54  Rachel Baker
   128  Ryan McCue
     4  Taylor Lovett
     1  jeremyfelt
     1  pkevan

Version 1.2

We’ve already started work on 1.2, and as always, we’re looking for help!

With version 1.2 and onwards, we’ll be tackling a bunch of extra testing for our endpoints, with the aim of eventually reaching >90% coverage. As always, we’ll also be adding new features and fixing bugs.

We’re also working on improving the new documentation site, and expect to see the majority of documentation migrated over there. Thanks to Sarah Gooding for helping out on the documentation side.

Core Integration

In case you missed it, the API is now slated for integration in WordPress 4.1. WP Tavern has a great writeup on the details.

As always, we look forward to seeing you at the team o2 and on GitHub. Now’s also a great time to remind you that you can get support for the plugin on WP.org, or by tweeting at me. Thanks to everyone who made this release great, and thanks to everyone using the plugin!

#json-api, #rest-api

JSON REST API: Version 1.0

I’m incredibly excited to announce the availability of version 1.0 of the JSON REST API.

This version is a huge release and introduces a bunch of new features, such as user, revision and post meta endpoints. It also introduces our long-term backwards compatibility policy, aligning with WordPress core backwards compatibility.

Here’s a selection of the new stuff:

  • Add user endpoints.

    Creating, reading, updating and deleting users and their data is now possible
    by using the /users endpoints. /users/me can be used to determine the
    current user, and returns a 401 status for non-logged in users.

    Note that the format of post authors has changed, as it is now an embedded
    User entity. This should not break backwards compatibility.

    Custom post types gain this ability automatically.

    (props @tobych, @rmccue, #20, #146)

  • Add post meta endpoints.

    Creating, reading, updating and deleting post meta is now possible by using
    the /posts/<id>/meta endpoints. Post meta is now correctly embedded into
    Post entities.

    Meta can be updated via the Post entity (e.g. PUT to /posts/<id>) or via
    the entity itself at /posts/<id>/meta/<mid>. Meta deletion must be done via
    a DELETE request to the latter.

    Only non-protected and non-serialized meta can be accessed or manipulated via
    the API. This is not predicted to change in the future; clients wishing to
    access this data should consider alternative approaches.

    Custom post types do not currently gain this ability automatically.

    (props @attitude, @alisspers, @rachelbaker, @rmccue, @tlovett1, @tobych,
    @zedejose, #68, #168, #189, #207)

  • Add endpoint for deleting a single comment.

    Clients can now send a DELETE request to comment routes to delete
    the comment.

    Custom post types supporting comments will gain this ability automatically.

    (props @tlovett1, @rmccue, #178, #191)

  • Add endpoint for post revisions.

    Post revisions are now available at /posts/<id>/revisions, and are linked in
    the meta.links.version-history key of post entities.

    Custom post types supporting revisions will gain this ability automatically.

    (props @tlovett1, #193)

  • Respond to requests without depending on pretty permalink settings.

    For sites without pretty permalinks enabled, the API is now available from
    ?json_route=/. Clients should check for this via the autodiscovery methods
    (Link header or RSD).

    (props @rmccue, #69, #138)

  • Add register post type argument.

    Post types can now indicate their availability via the API using the
    show_in_json argument passed to register_post_type. This value defaults to
    the publicly_queryable argument (which itself defaults to the
    public argument).

    (props @iandunn, @rmccue, #145)

  • Remove basic authentication handler.

    This breaks backwards compatibility for clients using Basic
    authentication. Clients are encouraged to switch to using OAuth
    . The Basic Authentication plugin can be
    installed for backwards compatibility and local development, however should
    not be used in production.

    (props @rmccue, #37, #152)

  • Require nonces for cookie-based authentication.

    This breaks backwards compatibility and requires any clients using cookie
    authentication to also send a nonce with the request. The built-in Javascript
    API automatically handles this.

    (props @rmccue, #177, #180)

  • Clean up deprecated methods/functions.

    Functions and methods previously deprecated in 0.8/0.9 have now been removed.
    Future deprecations will take place in the same manner as WordPress core.

    This breaks backwards compatibility, however these were marked as
    deprecated in previous releases.

    (props @rmccue, #187)

  • Only expose meta on ‘edit’ context as a temporary workaround.

    Privacy concerns around exposing meta to all users necessitate this change.

    This breaks backwards compatibility as post meta data is no longer
    available to all users. Clients wishing to access this data should
    authenticate and use the edit context.

    (props @iandunn, @rmccue, #135)

  • Add json_ensure_response function to ensure either a
    WP_JSON_ResponseInterface or a WP_Error object is returned.

    When extending the API, the json_ensure_response function can be used to
    ensure that any raw data returned is wrapped with a WP_JSON_Response object.
    This allows using get_status/get_data easily, however WP_Error must
    still be checked via is_wp_error.

    (props @rmccue, #151, #154)

  • Use version option to check on init if rewrite rules should be flushed.

    Rewrite rules on multisite are now flushed via an init hook, rather than
    switching to each site on activation.

    (props @rachelbaker, #149)

As always, you can view all commits or the longer changelog.

For those interested, here’s the list of contributors to this release:

$ git shortlog --summary 0.9...
     1  Chris Marslender
     1  Eric Lanehart
     2  K.Adam White
     1  Kat Hagan
     2  Matth_eu
    41  Rachel Baker
   139  Ryan McCue
     5  Taylor Lovett
    10  Toby Champion

There’s a few really important things to note with this release.

Authentication Changes

Authentication has changed significantly in 1.0. If you’ve been using Basic authentication previously, you’ll now need to install the Basic authentication plugin. This plugin is designed for local development, as Basic authentication requires sending your plaintext credentials over the wire, which is unsafe for production.

Production users have two choices: built-in cookie authentication, or OAuth authentication. OAuth 1.0a is an authorization protocol that allows you to authorize clients to act on your behalf, and does not require giving your username and password to the client. It does, however, require a significantly more complicated authentication/authorization process, and clients are required to register on the site beforehand. We’re working on long-term solutions to this.

Plugins and themes can also use built-in cookie authentication. This is the normal WordPress login process, however requires a nonce for authentication to the site. This is automatically handled for you when using the built-in Javascript client.

Backwards Compatibility

From this release forwards, backwards compatibility will not be broken. This includes both the internal PHP API, as well as the REST API we expose. New endpoints may be added, as well as new data, but responses will continue to be supersets of the current response.

The exception to this is for security concerns. As we continue development, we may need to change some endpoints for security issues, as we did with post meta for this cycle. These will be announced well before release where possible.

Please note also that this release has removed some previously deprecated methods and changed some internal implementation details. This only affects plugins or themes that extend the API.

Core Integration

We’re pushing hard for integration into 4.0 this cycle, and we need your help. Our core integration plan outlines the motivation behind the project and the specific plan for integrating it into core. We’re currently working on a comparison document for the API compared to the WP.com/Jetpack API and others, and will be publishing that soon.

We need your help to make it into 4.0. Developers, we’d love to know what you’ve built with the API, whether public or internal (even vague details help!), and we’d especially love to see things built with the API. We’re currently in danger of not making it in this cycle, so anything you can do to help us here would be fantastic.

As always, we’re also looking for help. The main API always needs help, and the other related projects do too. Both the WP-CLI client and Javascript client need help here.

You’re always welcome over at the team o2, and our next meeting will be at Tuesday, 00:00 UTC; we’d love to see you there. If not, see you soon for version 1.1!

#json-api, #rest-api

JSON REST API: Meeting Time Change

As announced on the team o2, the meeting time for the API team is changing to Tuesday, 0:00 UTC, in #wordpress-dev.

This week we’re discussing OAuth (which, P.S., is now available for testing), 1.0 and custom post type data privacy.

Hopefully I’ll see you there!

#json-api, #rest-api