Make WordPress Core

Updates from October, 2016 Toggle Comment Threads | Keyboard Shortcuts

  • Ryan McCue 4:05 am on October 8, 2016 Permalink |
    Tags: , , merge-proposals,   

    REST API Merge Proposal, Part 2: Content API 

    Hi everyone, it’s your friendly REST API team here with our second merge proposal for WordPress core. (WordPress 4.4 included the REST API Infrastructure, if you’d like to check out our previous merge proposal.) Even if you’re familiar with the REST API right now, we’ve made some changes to how the project is organised, so it’s worth reading everything here.

    (If you haven’t done so already, now would be a great time to install the REST API and OAuth plugins from WordPress.org.)

    A brief history of the REST API

    The REST API was created as a proof-of-concept by Ryan McCue (hey, that’s me!) at the WordPress Contributor Summit in 2012, but the project kicked off during the 2013 Google Summer of Code. The end result was Version 1.0, which grew into a community supported initiative that saw adoption and provided for a solid learning platform. The team used Version 1 to test out the fundamental ideas behind the API, and then iterated with Version 2, which made some major breaking changes, including explicit versioning, the introduction of namespacing for forwards compatibility, and a restructure of the internals. Version 2 also led to the infrastructure of the REST API being committed to WordPress core in 4.4.

    This infrastructure is the core of the REST API, and provides the external interface to send and receive RESTful HTTP requests. Since shipping in 4.4, the infrastructure is now used by WordPress Core for oEmbed responses, and by plugins like WooCommerce and Jetpack, enabling anyone to create their own REST API endpoints.

    The team has also been hard at work on the API endpoints. This has included core changes to WordPress to support the API, including deeper changes to both settings and meta.

    Today the REST API team is proposing the inclusion of a collection of endpoints that we term the “Content API” into WordPress Core.

    Proposals for Merge

    Content Endpoints

    For WordPress 4.7 the API team proposes to merge API endpoints for WordPress content types. These endpoints provide machine-readable external access to your WordPress site with a clear, standards-driven interface, allowing new and innovative apps for interacting with your site. These endpoints support all of the following:

    • Content:
      • Posts: Read and write access to all post data, for all types of post-based data, including pages and media.
      • Comments: Read and write access to all comment data. This includes pingbacks and trackbacks.
      • Terms: Read and write access to all term data.
      • Users: Read and write access to all user data. This includes public access to some data for post authors.
      • Meta: Read and write access to metadata for posts, comments, terms, and users, on an opt-in basis from plugins.
    • Management:
      • Settings: Read and write access to settings, on an opt-in basis from plugins and core. This enables API management of key site content values that are technically stored in options, such as site title and byline.

    This merge proposal represents a complete and functional Content API, providing the necessary endpoints for mobile apps and frontends, and lays the groundwork for future releases focused on providing a Management API interface for full site administration.

    Content API endpoints support both public and authenticated access. Authenticated access allows both read and write access to anything your user has access to, including post meta and settings. Public access is available for any already-public data, such as posts, terms, and limited user data for published post authors. To avoid potential privacy issues we’ve taken pains to ensure that everything we’re exposing is already public, and the API uses WordPress’ capability system extensively to ensure that all data is properly secured.

    Just like the rest of WordPress, the Content API is fully extensible, supporting custom post meta, as well as allowing more complex data to be added via register_rest_field. The API is built around standard parts of WordPress, including the capability system and filters, so extending the API in plugins should feel as familiar to developers as extending any other part of WordPress.

    This Content API is targeted at a few primary use cases, including enhancing themes with interactivity, creating powerful plugin interfaces, building mobile and desktop applications, and providing alternative authoring experiences. We’ve been working on first-party examples of these, including a mobile app using React Native and a liveblogging web app, as well as getting feedback from others, including WIRED, the New York Times, and The Times of London. Based on experience building on the API, we’ve polished the endpoints and expanded to support settings endpoints, which are included as the first part of the Management API.


    The API Infrastructure already in WordPress core includes support for regular cookie-based authentication. This is useful for plugins and themes that want to use the API, but requires access to cookies and nonces, and is hence only useful for internal usage.

    To complement the Content Endpoints, for WordPress 4.7 the API team also proposes merging the REST API OAuth 1 server plugin into WordPress Core. This plugin provides remote authentication via the OAuth 1 protocol, allowing remote servers and applications to interact securely with the WordPress API.

    OAuth is a standardised system for delegated authorisation. With OAuth, rather than providing your password to a third-party app, you can authorise it to operate on your behalf. Apps are also required to be registered with the site beforehand, which gives site administrators control over third-party access. Access to these apps can be revoked by the user if they are no longer using the app, or by a site administrator. This also allows apps with known vulnerabilities to have compromised credentials revoked to protect users.

    We’ve chosen OAuth 1 over the newer OAuth 2 protocol because OAuth 1 includes a complex system for request signing to ensure credentials remain secure even over unsecured HTTP, while OAuth 2 requires HTTPS with a modern version of TLS. While it is strongly encouraged for sites to use HTTPS whenever possible (Let’s Encrypt makes it easier than ever to do so), WordPress itself does not require HTTPS and we do not believe WordPress should make HTTPS a requirement for using the API. The additional complexity that OAuth 1 adds can be easily supported by a library, and many such libraries already exist in most programming languages. OAuth 1 remains supported around the web, including for the Twitter API, and we also provide extensive documentation on using it.

    Authentication Beyond 4.7

    One issue with OAuth over direct username and password authentication is that it requires applications to be registered on the site. For centralized OAuth servers this wouldn’t be a problem, but the distributed nature of WordPress installations makes this tough to handle: your application must be independently registered with every WordPress site it connects to. If you’ve ever had to create a Twitter or Facebook app just to use an existing plugin on your site, you’ll know this can be a less-than-optimal experience for users.

    To solve this distribution problem, we’ve created a solution called brokered authentication. This allows a centralised server (called the “broker”) to handle app registration and to vouch for these apps to individual sites. It simplifies app registration by allowing app developers to register once for all sites, and improves security by allowing the broker to vet applications and revoke them across the entire network. The system is designed to allow multiple brokers; while the main broker is run at apps.wp-api.org, organisations can run their own broker for internal usage, and developers can run a broker locally for testing.

    While the broker system has been running live at apps.wp-api.org for months, we want to stay conservative in our approach to the API, especially where security is concerned. We are therefore proposing brokered authentication for WordPress 4.8 to ensure we have further time to continue testing and refining the broker system. In addition, this will require an installation of the broker on a centralised server to act as the canonical broker for out-of-the-box WordPress. While apps.wp-api.org is currently acting in this role, this is currently hosted by a third-party (Human Made) on behalf of the API team. For long-term usage the broker should instead be hosted on WordPress.org, alongside the existing plugin and theme repositories. This migration will take time but we remain committed to continuing to develop and support the broker.

    After Merge

    After merging the REST API, the team plans to continue developing the API as before. We expect that integrating the REST API into WordPress core will bring additional feedback, and we plan on incorporating this feedback through the rest of the 4.7 cycle.

    During the remaining parts of this release cycle and through into the 4.8 cycle, additional work will go into other parts of the API. This includes further work and refinement on the broker authentication system, including work on WordPress.org infrastructure. Additionally, we plan to continue working on the Management API endpoints, including theme and appearance endpoints to support the Customiser team. Both of these components will be maintained as separate feature projects on GitHub until they’re ready for merge into core.

    The team remains committed to supporting the API in core, and the Content API will switch from GitHub to Trac for project management and contributions. This same process occurred for the API Infrastructure in WordPress 4.4.

    Reviews and Feedback

    With this merge proposal, we’re looking for feedback and review of the project. In particular, we’re focussing on feedback on the security of the API and OAuth projects, and are also reaching out to specific people for reviews. (We take the security of the API seriously, and bug reports are welcomed on HackerOne at any time.) Design and accessibility reviews for the OAuth authorisation UI are also welcomed to ensure we maintain the high standards of WordPress core.

    Both the REST API plugin and the OAuth plugin are available on WordPress.org, and issues can be reported to the GitHub tracker for the API and the OAuth plugin respectively. We have released a final beta (Beta 15 “International Drainage Commission”) which includes the meta and settings endpoints.

    With Love from Us

    As always, this is a merge proposal, and is not final until 4.7 is released. We’re eager to hear your thoughts and feedback; the comments below are a perfect place for that, or you can pop along to one of our regular meetings. We’re also always available in the #core-restapi room on Slack.

    We’d like to thank every single one of our contributors, including 88 contributors to the main repository and 23 contributors to the OAuth repository. Particular thanks goes to my (@rmccue) wonderful co-lead Rachel Baker (@rachelbaker), our 2.0 release leads Daniel Bachhuber (@danielbachuber) and Joe Hoyle (@joehoyle), and our key contributors for the 4.7 cycle: Adam Silverstein (@adamsilverstein), Brian Krogsgard (@krogsgard), David Remer (@websupporter), Edwin Cromley (@chopinbach), and K. Adam White (@kadamwhite). Thanks also to the core committers helping us out through the 4.7 cycle, including Aaron D. Campbell (@aaroncampbell) and Aaron Jorbin (@aaronjorbin), and to the fantastic release lead, Helen Hou-Sandí (@helen).

    Thanks also to everyone who has used the REST API, and to you for reading this. We built the REST API for you, and we hope you like it.

    With love, The REST API Team

    • George Stephanis 12:17 pm on October 8, 2016 Permalink | Log in to Reply

      Regarding Authentication, as nice as OAuth 1.0a may be in some respects, I feel that we’re missing some rather obvious problems here.

      We’re expressing such concern about securing the REST API endpoints to work on HTTP — that we’re missing the fact that for sites that don’t support HTTP, normal wp-login requests or cookies could just as easily be sniffed.

      It just feels to me like we’re spending a lot of time reinforcing the back door against attackers, when the front door isn’t even locked.

      The concept of a third-party broker feels very antithetical to WordPress Core. To have to ping the third-party server for every login to check for invalidations of their applications, let alone the initial confirmation of an application … for me, it doesn’t pass the gut check.

      I’d much rather see something akin to the Application Password system we’ve devised. It has unique per-application passwords, with 142 bits of entropy, usage logging (date and ip last used), and trivial invalidation, as well as a proposed simple flow for applications to request passwords and get the generated passwords passed back.. And, as an added bonus, it works with the legacy XML-RPC API.

      Compared to that, making someone hoping to do something for API access to a single site register for a cloud broker, and then reauth with the individual site .. the UX doesn’t pass muster for me. :\

      Authentication is probably the most important key to getting the REST API correct, and it’s necessary (imho) to ship. We need to get it right.

      • marsjaninzmarsa 7:25 am on October 9, 2016 Permalink | Log in to Reply

        To have to ping the third-party server for every login to check for invalidations of their applications, let alone the initial confirmation of an application … for me, it doesn’t pass the gut check.

        Nope, you’re missing the point, third party will be pinged only on first use of new app on particular site, and next every few hours or so for invalidation (same as with checking for security updates).

      • Dion Hulse 10:40 am on October 9, 2016 Permalink | Log in to Reply

        Core. To have to ping the third-party server for every login to check for invalidations of their applications, let alone the initial confirmation of an application … for me, it doesn’t pass the gut check.

        I tend to agree here.

        I’d much rather see something akin to the Application Password system we’ve devised

        However, I feel that Application Passwords are actually a far worse solution to the problem than the various oAuth options out there, to the point that I think Application Passwords are an anti-pattern which core should never directly support. The user experience of Application Passwords are not user friendly at all, they’re a hacky solution to avoid using a proper modern application authentication mechanism.

        I could ramble for quite some time on the UX of these, but at the end of the day moving towards oAuth is going to provide a far better developer and user experience for API clients.
        Unfortunately though with all options here there’s the downside that Applications need to be registered/known/considered safe and also be able to invalidate their credentials in the event the applications key store is leaked (For example, Plugin X which relies upon a service has it’s keystore hacked, the application would have no way of informing those sites to distrust it’s previous authenticated sessions).

        In an ideal world, a central provider wouldn’t be needed, but we don’t have a decentralised platform for WordPress yet, so there’s no other mechanism for WordPresses out there to be told the sort of information they need to know.

      • Ryan McCue 8:40 pm on October 9, 2016 Permalink | Log in to Reply

        At its core, OAuth is just a standardised system for issuing app tokens. The primary things that it builds on top of this are specifically for security over HTTP, so I would be extremely reluctant to move backwards from those to a more simple token-based system.

        Regular WordPress logins are protected by two main factors: nonces and session tokens. While it is possible to sniff a raw username/password over the initial login, subsequent requests are protected, which means automated attacks are much harder. The nature of an external API means neither WP nonces nor session tokens (which are both intentionally server-only secrets) can be used to protect authentication, so we need a replacement system.

        OAuth incorporates a combination of nonce and timestamp to replace these in a similar manner, which are then combined with the secret, which we need since it’s no longer server-only. (WP nonces and session tokens have an internal secret with NONCE_SALT, the only difference is that we share them with the client for OAuth.)

        The broker system exists specifically because dynamic registration (either with OAuth or your plugin) has a number of key problems, primarily that it makes revocation useless. If the app is revoked, it can simply register again, which subverts both the site admin and W.org team if it were intentionally revoked. We looked at dynamic registration and rejected it specifically because of the issues with it.

        While the system might seem conceptually complex, it’s pretty simple to actually implement, because the broker is just regular OAuth under the hood. The entirety of the flow can be written in 200 lines of JavaScript. From the app’s perspective, it primarily Just Works, with most of the complexity hidden inside the broker implementation instead.

        The OAuth 1.0a specification is rigorously tested by security researchers and in production on huge web apps. The broker system builds on top of this, but avoids reimplementing the wheel, and instead just uses OAuth for some of the heavy lifting. Application Passwords is at its core HTTP Basic Authentication, which in its own specification 17 years ago recommended against actually using it.

        OAuth 1 isn’t perfect, but every thing in it is there for a reason. Is it perfect? No. But it provides a lot of benefit over Basic Auth for a relatively low cost.

    • Jon Brown 1:09 pm on October 8, 2016 Permalink | Log in to Reply

      Much love back at you all. This looks fabulously well thought out and complete.

    • Josh Pollock 2:42 pm on October 8, 2016 Permalink | Log in to Reply

      5 Stars!

    • fgilio 3:09 pm on October 8, 2016 Permalink | Log in to Reply

      Thanks to all of you!

    • Ahmad Awais 4:07 pm on October 8, 2016 Permalink | Log in to Reply

      I’m all in. Let’s do it this time. BTW great read Ryan.

    • Omaar Osmaan 5:55 pm on October 8, 2016 Permalink | Log in to Reply

      Thanks so much for all the hard work from the team- eager to see it get merged on core!

    • Noel Tock 7:56 pm on October 8, 2016 Permalink | Log in to Reply


    • Sergio Santos 1:45 pm on October 9, 2016 Permalink | Log in to Reply

      Thanks to all the team behind this outstanding work!

    • Matthew Eppelsheimer 9:43 pm on October 9, 2016 Permalink | Log in to Reply

      Thumbs up!

    • Weston Ruter 12:12 am on October 10, 2016 Permalink | Log in to Reply

      Additionally, we plan to continue working on the Management API endpoints, including theme and appearance endpoints to support the Customiser team. Both of these components will be maintained as separate feature projects on GitHub until they’re ready for merge into core.

      I’d like to note that customizer changesets (#30937, formerly “transactions”) allows for REST API responses to have customizations applied when the request includes a customize_changeset_uuid query param which identifies a customize_changeset post. This allows the customizer to be used with headless sites and mobile apps.

      A feature plugin needs to be started shortly that allows for these changeset posts to be CRUD’ed via the REST API as well, allowing for new customizer administrative applications to be built on top of the REST API without any need to go to customize.php.

    • Tammie Lister 8:08 pm on October 10, 2016 Permalink | Log in to Reply

      As requested in the design Slack channel, here is a design review. I took the OAuth plugin for a spin today and only focused on the design elements.

      I first of all had discoverability issues, but that was mainly due to not using this. I sort of imagine if someone is going to use this they will be doing so knowing more than I did – so that’s ok.

      The first screen is minimal but as expected when empty. It seemed to all fit the existing WP patterns. Thanks for taking the time and doing that. Should there be select all boxes though – I don’t see any actions for all.

      On the ‘add’ screen, I tried adding nothing in the fields. I was surprised when got a green message. I would expect this should be red over the success green.


      The term ‘Add Application’ but then the action button being ‘Save Consumer’, this threw me. Perhaps this is my lack of knowledge over terminology, but shouldn’t they both be the same?

      The page gets cluttered fast, I sort of want a break here visually, just a simple line would do:


      Regenerating the secret success message I felt should be by the secret. It was odd as the message appeared above my eye view. I click to regenerate lower and then this message pops up the top, feels like it should be close to add context.


      This may be a bug but when I searched for applications it tried to search users and I got the following:


    • Matt Mullenweg 8:26 pm on October 10, 2016 Permalink | Log in to Reply

      1. Given the hurdles of authentication I don’t think that bringing this into core provides benefits for WP beyond what the community gets from the plugin. I don’t believe in its current state the benefit outweighs the cost, and we should err on the side of simplicity.

      2. I am not interested in hosting the centralized brokered authentication server on WordPress.org in the 4.8 timeframe, and hesitant about the implications it has for WP more broadly. I do appreciate the thought that has been put into solving this tricky problem.

      • K.Adam White 9:39 pm on October 11, 2016 Permalink | Log in to Reply

        @matt, thanks for the comment. Can you clarify whether with (1) you mean the authentication solution, or the merge proposal for the content endpoints more broadly? There was discussion today in slack where we weren’t sure which way to interpret this point.

    • Dominik Schilling (ocean90) 10:01 pm on October 10, 2016 Permalink | Log in to Reply

      I gave the OAuth 1 plugin a try and here’s some feedback:

      • https://github.com/WP-API/OAuth1/issues/161 makes me wonder how we can integrate something with may not work on “a lot of sites”.
      • Adding Applications as a sub menu of Users doesn’t feel right to me. I just can’t imagine that 80% of our users really need this there. Twitter and Facebook have it both in the settings.
      • The UI: https://cloudup.com/c1emHXcHP4g Reusing existing elements is fine, but that looks a bit to simple and not fully developed. I’m sure that this can made better and doesn’t require a table. See Twitter/Facebook.
      • The plugin is using closures so it’s not compatible with the minimum requirements of WordPress.
      • I got a “500 Internal Server Error” error when the callback URL was invalid.
      • A few DocBlocks don’t follow the documentation standards.

      For some reasons I can’t get a successful `oauth1/access` request, I’ll try this again tomorrow…

    • Aaron D. Campbell 7:30 pm on October 11, 2016 Permalink | Log in to Reply

      The API endpoints themselves seem to be in a really good place, but I’m not sure the oAuth plugin is. Would it be reasonable to separate these into different proposals?

    • Mark Howells-Mead 2:32 pm on October 12, 2016 Permalink | Log in to Reply

      Issues aside, I’m strongly in favour of the REST API becoming a core feature as soon as it is stable.

    • Chris Smith 3:13 pm on October 12, 2016 Permalink | Log in to Reply

      +1 here as well and thanks to the team for their hard and thoughtful work.

      Based on the comments here, it does seem like it would make sense to split the proposal into the Content API and oAuth pieces for 4.7 so they can be dealt with/commented on independently as needed.

      @kadamwhite, what were the team’s thoughts on that in yesterday’s discussion?

    • Nilambar Sharma 5:32 pm on October 12, 2016 Permalink | Log in to Reply

      Really waiting for this. Lets make WordPress more awesome. It would be a great step merging new endpoints to core.

    • brad989 5:33 pm on October 12, 2016 Permalink | Log in to Reply

      Been looking forward to this for a long time. The content endpoints are an essential addition to WordPress core.

    • Chuck Reynolds 6:52 am on October 14, 2016 Permalink | Log in to Reply

      full api integration into core needs to happen eventually… API endpoints are at/near a good place. re: oath tho… meh..
      needs full support of CPT’s, which theoretically it would but… lots to test there.

    • forbze 7:16 am on October 14, 2016 Permalink | Log in to Reply

      I really want this! I’ve had to use hacky solutions in the past to repurpose content for mobile apps (I didn’t want to use web views / Mobile ui plugins).

      WordPress’s admin interface would make it really simple for me to give clients content control for their app content – rather than me having to build a custom editor UI on top of Parse / Firebase.

      I think oauth is the right solution for auth.

    • pwaring 9:23 am on October 14, 2016 Permalink | Log in to Reply

      I’m only in favour of this if the API can be disabled (ideally by default). I don’t need or want this functionality and it sounds like another remote attack vector.

      “we do not believe WordPress should make HTTPS a requirement for using the API”

      I *strongly* disagree with this. TLS uptake is increasing all the time and some browsers (e.g. Chrome) are starting to show stronger warnings for plain HTTP. Plus, if a popular application like WordPress requires HTTPS for its API, that will act as a significant nudge to hosting providers to provide HTTPS by default (a bit like moving from PHP 4 to 5).

      • Ryan McCue 9:18 pm on October 14, 2016 Permalink | Log in to Reply

        It’s super easy to disable the REST API via the `rest_enabled` filter (just return false), and there’s already a plugin for it. 🙂

        • pwaring 8:10 am on October 15, 2016 Permalink | Log in to Reply

          Whilst I’m capable of changing the code to add a filter, I don’t think that counts as ‘super easy’. It really should be a tick box somewhere in the admin settings.

          Installing a plugin to disable optional behaviour in core also seems unnecessarily complicated, plus it introduces another dependency and there’s no guarantee that the plugin author will keep it up to date.

          • Dave McHale 1:59 pm on October 18, 2016 Permalink | Log in to Reply

            I understand your concern, pwaring, but we already don’t have the ability to disable XML-RPC in core (ever since they enabled it by default in 3.5) – and since the REST API is “the same but different” I believe that’s why the new feature is following the same model. “Decisions, not options” and all that 🙂

            It’s also why I’m the author of the plugin Ryan linked to above. I think the REST API is great, and can’t wait to start using it myself *and* also see what other people build with it. But it’s not going to be for everyone, and I thought that people may want the easy ability to turn it off if they had a reason to.

            Since the plugin uses the root-level filters provided by the REST API there should very, VERY little need for actual plugin updates moving forward into the future (though I do keep an eye on the project, in case an update is needed). The only time I’ve needed to make a real functional change was when the filters themselves changed from version 1.x to 2.x of the API, before it was even merged into core. New features like the content endpoints won’t change the use of the plugin, because the REST API is still entirely disabled if you’re using the filters.

  • John Blackbourn 3:40 pm on October 3, 2014 Permalink

    WordPress 4.1 development is under way. See the release schedule for the key dates.

    Here’s an overview of the headline features for this release. If you’d like to get involved with mockups, design, development, documentation, testing – anything really – see the users and tickets listed below each item.

    Other areas which will receive attention during this release:

    • daveshine (David Decker) 8:40 am on October 4, 2014 Permalink | Log in to Reply

      This is actually a great list, execpt for the “focus” thing (have to see it live for myself) I totally welcome all improvements here personally. Really looking forward to it! 🙂
      Thank you all for your hard work on this release cycle!

    • Nashwan Doaqan 7:29 pm on October 4, 2014 Permalink | Log in to Reply

      Nice! .. Thank u for the hard work 🙂

    • Gary Pendergast 4:33 am on October 7, 2014 Permalink | Log in to Reply

      Improved UI for installing and updating plugins and themes (async installation), possibly core too.

      Tracking ticket added: #29820

    • Christian Foellmann 3:08 pm on October 8, 2014 Permalink | Log in to Reply

      Great stuff to look forward to.
      If you are evaluating Select2 to be included in core (was hoping for that a long time) you might want to talk to the maintainers/devs on what changes Select2 4.x will bring.

compose new post
next post/next comment
previous post/previous comment
show/hide comments
go to top
go to login
show/hide help
shift + esc
Skip to toolbar