Improving the REST API users endpoint in multisite

One of the objectives for multisite is sorting how users are managed with the REST API. This was one of the agenda items for last week’s #core-multisite office hours and generated some good discussion. Here’s a wrap-up of the ideas and thoughts from that discussion.

Chat log in #core-multisite

Attendees: @iamfriendly, @johnjamesjacoby, @nerrad, @florian-tiar, @mikelking, @earnjam, @jeremyfelt

Users in multisite exist globally and are shared among sites on one or more networks. Users are associated with sites in the user meta table with a wp_#_capabilities key.

The current state of the wp-json/wp/v2/users endpoint for multisite is:

  • A POST request for a new global user to the main site creates the user and associates them with the main site only.
  • A POST request for a new global user to a sub site creates the user and associates them with the sub site only.
  • A POST request for an existing global user results in an error.
  • A PUT request for an existing global user to a sub site updates the user’s meta with a capability for that sub site.
  • A DELETE request on multisite is invalid and results in an error. See #38962.

It is not possible to remove a user from an individual site or to delete the user from the network.

Previous tickets: #38526, #39155, #38962, #39000

The following are a few thoughts expressed separately from the above summary.

  • The right way to associate existing objects over the REST API is with a PUT request.
  • The right way to disassociate existing objects is with a PUT request.
  • Linked previous discussion – “Deleting an item should always delete an item
  • We already have functions like remove_user_from_blog() and add_user_to_blog() available to us.
  • Does “add” invite or literally add? This can probably be included as data in the PUT request.
  • What happens if an API client is built for single site and then that site gets switched to multisite?
  • Handling bulk actions on an endpoint would be nice. (e.g. Add a user to multiple sites) No endpoint has implemented batch handling yet though.

Initial tasks:

  • It should be possible to remove a user from a site with a PUT request to the wp-json/wp/v2/users/# endpoint.
  • It should be possible to delete a global user with a DELETE request to the wp-json/wp/v2/users/# endpoint once all sites have been disassociated.

New tickets will be created soon for these tasks. Please leave any initial feedback in the comments on this post covering the assumptions and conclusions made above. There will be another round of discussion during tomorrow’s #core-multisite office hours at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

#multisite, #networks-sites, #rest-api, #users

Focus Tech and Design Leads

There are three main focuses this year: the REST API, the editor, and the customizer.

For the REST API we’re going to work on getting first party wp-admin usage of the new endpoints, and hopefully replace all of the core places where we still use admin-ajax.

The editor will endeavour to create a new page and post building experience that makes writing rich posts effortless, and has “blocks” to make it easy what today might take shortcodes, custom HTML, or “mystery meat” embed discovery.

The customizer will help out the editor at first, then shift to bring those fundamental building blocks into something that could allow customization “outside of the box” of post_content, including sidebars and possibly even an entire theme.

Each focus will have a tech lead, and a design lead, and I’ll be working closely with each to make sure we’re aligned and moving diligently in the right direction even though we don’t have the normal release hooks. These starting leads will be:

REST API: Ryan McCue and KAdam White.

Editor: Matias Ventura and Joen Asmussen.

Customizer: Weston Ruter and Mel Choyce.

Given there is no set timeline for releases that would normally set a term, these leads are free to bow out at any time they feel they can’t contribute fully and we’ll find a replacement.

You might be wondering what each lead is responsible for. The REST team gave some thought to this for their focus, and this is the list they came up with:

Tech Lead responsibilities:

  • identify and ensure implementation of first-class REST API usage within WP-Admin,
  • replacing/refactoring admin-ajax use.
  • overall REST API architecture
  • infrastructure and endpoint performance
  • security at an infrastructure and endpoint (data-exposure) level
  • improving authentication options and documentation
  • working with the Design Lead to build new, or expand on existing, endpoints
  • working with the Design Lead to address usability feedback for the infrastructure and endpoints

Design Lead responsibilities:

  • usability of endpoints for internal or external clients
  • usability of the infrastructure from the perspective of a API client
  • working with the Editor and Customizer focus teams to collect requirements and gather feedback
  • identifying ways to improve the overall experience for folks building clients or consuming endpoints (like documentation)

#customizer, #editor, #rest-api

REST API: Agenda for October 24 Meeting

Here’s the agenda for Monday’s weekly meeting for the REST API project, at 2016-10-24 14:00 UTC in the #core-restapi channel in slack. The meeting will run for one hour.

  • Discuss post-merge Process Changes & the future of the WP-API GitHub Repository
  • Review Trac tickets: what hasn’t been migrated from GitHub?
  • Clarify priorities & responsibilities leading up to Beta 1
  • Open floor.

Meetings are every Monday at 1400 UTC. See you there!

#4-7, #agenda, #rest-api

WordPress REST API Success Metrics

This post is part of the conditional merge of the WordPress REST API.

Tracking and measuring specific success metrics is a priority for the WordPress REST API Content Endpoints project. The aim is to be able to look back at the implementation of the feature project to see how it went after the fact, and to utilize that information to inform future decision making.

A submission form will be created to help track implementations, in addition to automated scripting and analysis. Feedback will be grouped based on which metric it applies to. Developers and implementers will be polled to determine usage, where automation is a challenge.

It is very important to note prior to analyzing individual goals, that these goals and the achievement of them does not automatically indicate success, or that not achieving them means a lack of success. For instance, one metric that’s very hard to capture in goals like those that follow is the impact of a singular implementation. One highly impactful implementation could indicate a great deal of success, but would not be fully captured by the following metrics. These goals are established to help identify success metrics that would offer clear insight on the project, but they are not all encompassing or definitive alone.

Metric #1: Plugin utilization

Per the “Popular” plugins page of the WordPress Plugin Directory, the following can be determined:

  • 21 plugins have 1m+ active installs
  • 126 plugins have 200,000+ active installs
  • 232 plugins have 100,000+ active installs

Many of these are not good candidates for utilizing the REST API, but there are a number of plugins that are.

The following goals are proposed:

  • 15% utilization of plugins with more than 1 million installs.
  • 7% utilization of plugins with more than 200,000 installs and fewer than 1 million.
  • 3% utilization of plugins with more than 100,000 installs and fewer than 200,000.

Additionally, it is proposed to seek 5-10 examples of non-directory or otherwise prominent plugins, that may not fit the above metrics, to showcase their utilization of the API, and how it has benefited them. This may include commercial plugins.

Timeline for analysis: three major releases after merge (5.0 release day).

Metric #2: Distributed theme utilization

Popular themes are a bit of a misnomer in the WordPress landscape. The default themes or themes that have been around for a very long time account for a significant percentage of all active themes. And feature development isn’t as common for existing themes as it is for existing plugins.

In addition to polling free theme makers, top theme shops (according to Alexa) should be polled to determine usage of the API for theming purposes.

Additionally, coordination can occur with the Theme Review Team to identify usage of the API in newly submitted themes, to determine adoption.

The API could be useful in themes themselves, as well as complimentary plugins, which are not likely to be picked up by the plugin analysis. However, it’s difficult to know exactly how WordPress themes may use the content endpoints in a distributed theme context. It could be part of efforts within the customizer, page building, custom widgets, etc.

A goal of 2% adoption of the API in new themes is proposed, determined within three major releases, and analyzed amongst distinct ecosystems:

  • New WordPress.org theme review submissions
  • Third party theme shops (A shop using the API in any capacity, likely new releases, would count as “yes”, rather than theme by theme)
  • Analysis of page builder or helper plugins
  • Distinct theme communities with “builder”, “framework”, or “starter” themes.

Metric #3: Custom development implementations

The WordPress REST API is immediately useful for custom development projects, where the developer has control over the environment. The REST API in core provides a vote of confidence to outside developers that they are building with an API that has the full backward compatibility and stability expected of features in the core project.

It is proposed to identify 50+ projects in production or under development utilizing the Content Endpoints, within two major release cycles.

The likelihood is that there will be far more consumption-only applications, versus those that manipulate content. Within the same two release timeframe, the goal is to identify at least 12 custom projects that utilize the Content Endpoints beyond a read-only use case.

Metric #4: Core development and feature projects

The WordPress REST API can aid core development and feature projects. Items like Quick Draft, Press This and a menu bar revisions browser are test cases for using the API in a core context with WordPress 4.7.

Utilization of the API for additional core features and feature projects will be tracked over the next three major release cycles. It is proposed to identify 6 core features or patches than can utilize the API, as well as 1 feature project, within three major releases.

#4-7, #rest-api

REST API Team Meeting Notes, 2016-10-17

As a reminder, in just 30 minutes there is a meeting in #core to decide whether the REST API Content Endpoints will be merged as a part of WordPress 4.7! See the ongoing discussion of this proposal here. Note that the OAuth server is no longer proposed for merge at this time, but authentication options will be a primary focus area for the API project during the 4.8 development cycle.

The REST API Content Endpoints provide a new foundation upon which the WordPress developer community can build themes, plugins, and core feature. They represent a common standard and consistent interface across WordPress’s core content data types, and provide robust support for custom post types and meta values. These endpoints lay the foundation upon which future releases will add remote authentication options, even deeper querying abilities, and broader endpoint coverage for site management. This iterative approach fits WordPress’s development model and philosophy, advancing the project’s long-term goal of opening WordPress up to a wider developer audience and helping to ensure continued work on the REST API in the release cycles and years to come.

Meeting Notes

At today’s weekly API team meeting in core-restapi (agenda here) the team resolved all outstanding decisions milestoned for the REST API 2.0 / WordPress 4.7 merge candidate:

  • The ?filter query parameter will be removed from the REST API plugin prior to core merge, a breaking change that improves the consistency of querying the API and eliminates a set of parameters that could introduce backwards compatibility issues were they to be committed to WordPress core. A separate plugin will be published to reinstate the `filter` parameter on a strictly opt-in basis.
  • Comments on password-protected posts are being deferred as a future enhancement until a robust solution is proposed that permits the API to adequately mirror existing functionality.
  • The unfiltered_html capability should be respected by the API, and a patch will be submitted to bring the API’s behavior in line with core’s.

There are 29 tickets left in the 2.0 milestone, several of which have open pull requests already. These issues represent a mix of outstanding bugs, documentation needs and improvements that will be moved to trac should the merge proposal be accepted.

The REST API team leads would like to recognize that the content endpoints plugin now has 95 contributors: thank you and welcome to all of the new participants who have joined the project in the past week!

#4-7, #meeting-notes, #rest-api

Merge Proposal Discussion: REST API Content Endpoints

There are discussion meetings and office hours in #core-restapi at 2016-10-14 14:00UTC and 2016-10-14 19:00UTC on Friday the 14th. Our next team meeting is on 2016-10-17 14:00UTC. Please attend some of all of these, because

We are meeting at 2016-10-18 01:00 UTC to make a decision on this merge proposal!

To that end, the below discussion points will be updated regularly, please leave comments on this post or join the conversation in #core-restapi.

Yesterday at the dev chat the API Team proposed the Content API Endpoints for merge in WordPress 4.7. There was popular support for this feature but as @jorbin and @helen noted that the lack of dissent suggested additional review is needed, so the API Team is continuing to seek thorough review & constructive criticism of the content endpoints, including the open questions previously shared on the week 7 and week 8 API team updates.

The merge proposal also engendered follow-up discussion in the #core-restapi channel about the benefit content endpoints bring to core, whether having such endpoints built in is quantifiably more beneficial than having them as a plugin, whether moving development from a plugin to core would slow development, and whether the endpoints as-proposed have been sufficiently reviewed for security and performance. We attempt to capture those questions & concerns (and the perspectives on them) below.

Security

Have the content API endpoints been thoroughly reviewed for security?

  • The REST API plugin has been on HackerOne for over a year with paid bounties for bugs
  • @barry has begun a security review

Performance

How does the API measure up against alternatives? Are there concerns about how the API could impact the servers to which it is deployed?

  • DeliciousBrains did a performance comparison with Admin AJAX and found the REST API to have a performance improvement (These tests have not yet been independently verified)
  • @mikeschroder notes in the comments that using the REST API in place of Admin-Ajax will also bring speed benefits by permitting many previously-uncacheable requests to be cached.

User Feedback

Are the content endpoints sufficiently well-tested & vetted by the community?

  • @matt questions whether feedback is coming too late in development for concerns to be acted upon
    • @rmccue notes that the v2 endpoints were created based on user feedback; REST API endpoints are being deployed by plugins and running on VIP, and the content endpoints have been in wide use across a variety of sites, leading to 90+ code contributors and far more developers’ support & feedback on the endpoints
  • @rmccue has also reached out to Phil Sturgeon for feedback and will follow up

Do Content Endpoints Benefit Core Development?

Will having these endpoints in core improve future core development, or solve any immediate problems?

  • @bradyvercher suggested that the content API endpoints would remove the need to write a variety of one-off ajax callbacks when developing future WordPress Core AJAX functionality
  • @westonruter notes that the customizer could dynamically create settings for posts and other kinds of content without having to wire up new admin-ajax handlers

Will Merging Negatively Impact API Development?

Will having to work through trac instead of GitHub cause development to atrophy?

  • @jjj argues that merging will slow development, because GitHub-hosted plugins are not bound to WordPress release cycles and have less overhead for features to be developed and deployed for testing. @jjj requested a plan for how the REST API will be developed going forward after the merge of these endpoints that would account for the added friction.
  • @krogsgard countered that core increases the visibility of a project like the content endpoints
    • The number of new contributors in this Slack discussion suggests that this merge proposal is bringing in new voices; whether this supports Brian’s point or not, the team is grateful for the breadth of perspectives being shared -Ed.
  • @rachelbaker suggested that the API endpoints are sufficiently inter-dependent with core WordPress code that maintaining the plugin separately amounts to maintaining a fork, and that such separated development is untenable long-term.
  • @matt hopes that a merge of these endpoints would slow release speed, but not development speed; @rmccue feels that development speed will stay the same or increase, and that tying releases to WordPress Core increases the stability and predictability of the API endpoints.
  • The versioning of the API endpoints supports forward compatibility

Do Content Endpoints Belong on Every WordPress Site?

What are the pros and cons to having every WordPress site have content API endpoints?

  • @rmccue suggests the API has network effects that can only be realized with a large install base. @krogsgard draws a comparison to RSS, the widespread availability of which enables everything from podcasting from WP to the use of apps like Feedly.
  • @matt suggests that the Atom API is a better analogue than RSS, which is an independent and pre-existing standard, and that network effects could be tested through inclusion in Jetpack
  • @joostdevalk notes that many plugins, like Yoast, have data tied to existing content such as posts and pages; either they expose the content through their own endpoints, or core does. If Core exposes content types through the API then plugins may build on top of that shared foundation, not independently reinvent the wheel. “if this doesn’t end up in core, we’ll start rolling our own API for stuff. Others will too. Interoperability won’t be there, for even the most basic stuff. I think this isn’t like RSS, I think this is much more like all of us using the same table space in MySQL.”
    • @shelob9 and @masonjames agree that merging the endpoints would create a consistent and reliable open “standard” that WordPress developers can use instead of continually reinventing how to read and edit post data over HTTP.
    • In response to the question “what prevents you from building on the endpoints in their plugin form,” @joostdevalk went on to note that plugin dependencies would make that a viable option, but that burden currently lies on the user. Plugin installation is not frictionless.
  • Can these endpoints be bundled? short takeaway: no
    • Woo bundled the API infrastructure before it was merged; doing so for content endpoints would require bundling prohibitively large amounts of endpoint code.
    • @nerrad worries that if plugins bundle different versions of the endpoints plugin, then those plugins may conflict if all bundled copies are not kept in sync.
      • @nerrad clarifies in the comments below that these worries also encompass the additional risk of conflicts when plugin authors each build their own versions of these content endpoints, instead of leveraging a shared standard: if two plugins each expose their own REST collection for posts, a developer working on a site with multiple such endpoints will need to decide which to extend, and will then have their extension tied to that specific plugin rather than to a core API.
  • @schrapel and @jorbin discussed that these content endpoints make it much easier to crawl a site, which also brings some potential performance concerns: no new content is exposed, but the process of aggregating it is easier and more easily automated.
  • In the  comments below @foliovision believes that merging the endpoints will be the best way to assert the level of back-compatibility that mid-size agencies need in order to confidently utilize the endpoints.

Please leave thoughts, questions, concerns, comments & experience in the comments below. Thank you!

Edited 2016-10-16 to include the below comments into the body of the post

#4-7, #rest-api

REST API Team Update, 4.7 Week 8

Summary: Beta 15 has been released, there are open questions that would benefit from your feedback, and the Content API Endpoints and OAuth Server are being proposed for merge as distinct, separate enhancements to the existing WordPress REST API infrastructure.

REST API v2 Beta 15 released

The 15th beta release of the REST API content endpoints plugin was released on October 7. This release builds on top of the recent Beta 14 to…

  • Add support for Post Meta, Term Meta, User Meta and Comment Meta within their parent endpoints
  • Introduce a settings endpoint to allow key site setting values to be retrieved & modified using the API
  • Introduce query parameters to query for posts that are NOT IN one or many terms of specific taxonomies
  • Resolve bugs, including bad comparison logic when updating comments.

Please try it out and report any outstanding issues; the REST API project gained its 90th code contributor this week and the team is deeply grateful for the energy and support of the broader WordPress community in testing out this merge-candidate plugin!

New Questions & Discussion Items

Items which have arisen through final ticket triage & review on which the team seeks feedback:

  1. Should the `filter` shim should be removed prior to merge? It is the majority position of the API team that `filter` be deprecated to dramatically improve the simplicity and consistency of API query functionality
  2. How should comments be handled for password-protected posts? Should the password be passed as a query parameter with the PUT/POST request, or is there a better option?
  3. Should the API match core’s logic when users with the `unfiltered_html` capability are creating or updating Posts or Comments?

Meeting Notes

At the weekly team meeting on October 10 the group reviewed open issues in the 2.0 milestone, which represents the candidate for our merge proposal shared last week.

Meeting attendees agreed to review open issues and pull requests individually, and to reconvene on Tuesday at 1500UTC to ensure all priority tickets had an owner.

At that meeting on October 11, the team reviewed the incoming feedback around the OAuth plugin (linked above). While the API team feels that having a built-in authentication solution provides a much-needed service, particularly to developers building mobile and desktop applications, the design and usability feedback we have received does indicate that the plugin needs more work.

OAuth’s place in the Merge Proposal

The API Team believes that the identified issues are resolvable, that the OAuth plugin is on track and that it should still be considered for merge in 4.7. However, after discussion within the team, input from @matt, and advice from @aaroncampbell and other core committers, we have edited our merge proposal to submit the Content API Endpoints and OAuth server as separate merge candidates. The API Team proposes both components for merge, but we submit the content endpoints for consideration independently of the OAuth1 server.

Content Endpoints Without OAuth

The Content API endpoints are stable, well-tested, and in wide production use across a variety of applications. Theme and plugin developers will benefit from having canonical, well-tested API endpoints in core, which may be used to query WordPress both from PHP code and from JavaScript applications running on the front-end or admin of WordPress. Sharing the endpoints for core data types enables increased consistency of what data is exposed and how it is persisted across different plugins, improving consistency and shortening development time by using . These themes and plugins have full read and write access to the API using the existing cookie & nonce authentication.

Mobile and desktop applications can leverage these same endpoints in a read-only capacity to create a variety of powerful reader-oriented applications and tools that expand the capability of what WordPress can do today, such as a unified reader for Make WordPress blogs and other experiments hypothesized by @jorbin.

Should OAuth 1 not be accepted for 4.7, secure write access for these external applications would still be only a plugin install away; and while having an OAuth server in core will provide a canonical approach for authenticating from remote applications, depending on the needs of a specific site or specific client application other authentication schemes may actually be preferable. Plugins exist for JWT Authentication and of course OAuth 2, and should OAuth 1 not be accepted for 4.7 these plugins may still be installed to enable an external application to opt-in to secure write access to your WordPress site.

In Summary

The API team submits for 4.7 merge consideration two enhancements to the REST API infrastructure: the Content API Endpoints for core WordPress datatypes, and an OAuth server which will reduce the setup time needed to securely interact with those endpoints from outside of WordPress. We believe these enhancements are each individually sufficiently tested and mature to meet the quality and security standards of WordPress Core, and each individually provides wide-reaching benefit to WordPress developers, and through them to the authors, readers & publishers of the web.

#4-7, #rest-api

REST API Merge Proposal, Part 2: Content API

Hi everyone, it’s your friendly REST API team here with our second merge proposal for WordPress core. (WordPress 4.4 included the REST API Infrastructure, if you’d like to check out our previous merge proposal.) Even if you’re familiar with the REST API right now, we’ve made some changes to how the project is organised, so it’s worth reading everything here.

(If you haven’t done so already, now would be a great time to install the REST API and OAuth plugins from WordPress.org.)

A brief history of the REST API

The REST API was created as a proof-of-concept by Ryan McCue (hey, that’s me!) at the WordPress Contributor Summit in 2012, but the project kicked off during the 2013 Google Summer of Code. The end result was Version 1.0, which grew into a community supported initiative that saw adoption and provided for a solid learning platform. The team used Version 1 to test out the fundamental ideas behind the API, and then iterated with Version 2, which made some major breaking changes, including explicit versioning, the introduction of namespacing for forwards compatibility, and a restructure of the internals. Version 2 also led to the infrastructure of the REST API being committed to WordPress core in 4.4.

This infrastructure is the core of the REST API, and provides the external interface to send and receive RESTful HTTP requests. Since shipping in 4.4, the infrastructure is now used by WordPress Core for oEmbed responses, and by plugins like WooCommerce and Jetpack, enabling anyone to create their own REST API endpoints.

The team has also been hard at work on the API endpoints. This has included core changes to WordPress to support the API, including deeper changes to both settings and meta.

Today the REST API team is proposing the inclusion of a collection of endpoints that we term the “Content API” into WordPress Core.

Proposals for Merge

Content Endpoints

For WordPress 4.7 the API team proposes to merge API endpoints for WordPress content types. These endpoints provide machine-readable external access to your WordPress site with a clear, standards-driven interface, allowing new and innovative apps for interacting with your site. These endpoints support all of the following:

  • Content:
    • Posts: Read and write access to all post data, for all types of post-based data, including pages and media.
    • Comments: Read and write access to all comment data. This includes pingbacks and trackbacks.
    • Terms: Read and write access to all term data.
    • Users: Read and write access to all user data. This includes public access to some data for post authors.
    • Meta: Read and write access to metadata for posts, comments, terms, and users, on an opt-in basis from plugins.
  • Management:
    • Settings: Read and write access to settings, on an opt-in basis from plugins and core. This enables API management of key site content values that are technically stored in options, such as site title and byline.

This merge proposal represents a complete and functional Content API, providing the necessary endpoints for mobile apps and frontends, and lays the groundwork for future releases focused on providing a Management API interface for full site administration.

Content API endpoints support both public and authenticated access. Authenticated access allows both read and write access to anything your user has access to, including post meta and settings. Public access is available for any already-public data, such as posts, terms, and limited user data for published post authors. To avoid potential privacy issues we’ve taken pains to ensure that everything we’re exposing is already public, and the API uses WordPress’ capability system extensively to ensure that all data is properly secured.

Just like the rest of WordPress, the Content API is fully extensible, supporting custom post meta, as well as allowing more complex data to be added via register_rest_field. The API is built around standard parts of WordPress, including the capability system and filters, so extending the API in plugins should feel as familiar to developers as extending any other part of WordPress.

This Content API is targeted at a few primary use cases, including enhancing themes with interactivity, creating powerful plugin interfaces, building mobile and desktop applications, and providing alternative authoring experiences. We’ve been working on first-party examples of these, including a mobile app using React Native and a liveblogging web app, as well as getting feedback from others, including WIRED, the New York Times, and The Times of London. Based on experience building on the API, we’ve polished the endpoints and expanded to support settings endpoints, which are included as the first part of the Management API.

Authentication

The API Infrastructure already in WordPress core includes support for regular cookie-based authentication. This is useful for plugins and themes that want to use the API, but requires access to cookies and nonces, and is hence only useful for internal usage.

To complement the Content Endpoints, for WordPress 4.7 the API team also proposes merging the REST API OAuth 1 server plugin into WordPress Core. This plugin provides remote authentication via the OAuth 1 protocol, allowing remote servers and applications to interact securely with the WordPress API.

OAuth is a standardised system for delegated authorisation. With OAuth, rather than providing your password to a third-party app, you can authorise it to operate on your behalf. Apps are also required to be registered with the site beforehand, which gives site administrators control over third-party access. Access to these apps can be revoked by the user if they are no longer using the app, or by a site administrator. This also allows apps with known vulnerabilities to have compromised credentials revoked to protect users.

We’ve chosen OAuth 1 over the newer OAuth 2 protocol because OAuth 1 includes a complex system for request signing to ensure credentials remain secure even over unsecured HTTP, while OAuth 2 requires HTTPS with a modern version of TLS. While it is strongly encouraged for sites to use HTTPS whenever possible (Let’s Encrypt makes it easier than ever to do so), WordPress itself does not require HTTPS and we do not believe WordPress should make HTTPS a requirement for using the API. The additional complexity that OAuth 1 adds can be easily supported by a library, and many such libraries already exist in most programming languages. OAuth 1 remains supported around the web, including for the Twitter API, and we also provide extensive documentation on using it.

Authentication Beyond 4.7

One issue with OAuth over direct username and password authentication is that it requires applications to be registered on the site. For centralized OAuth servers this wouldn’t be a problem, but the distributed nature of WordPress installations makes this tough to handle: your application must be independently registered with every WordPress site it connects to. If you’ve ever had to create a Twitter or Facebook app just to use an existing plugin on your site, you’ll know this can be a less-than-optimal experience for users.

To solve this distribution problem, we’ve created a solution called brokered authentication. This allows a centralised server (called the “broker”) to handle app registration and to vouch for these apps to individual sites. It simplifies app registration by allowing app developers to register once for all sites, and improves security by allowing the broker to vet applications and revoke them across the entire network. The system is designed to allow multiple brokers; while the main broker is run at apps.wp-api.org, organisations can run their own broker for internal usage, and developers can run a broker locally for testing.

While the broker system has been running live at apps.wp-api.org for months, we want to stay conservative in our approach to the API, especially where security is concerned. We are therefore proposing brokered authentication for WordPress 4.8 to ensure we have further time to continue testing and refining the broker system. In addition, this will require an installation of the broker on a centralised server to act as the canonical broker for out-of-the-box WordPress. While apps.wp-api.org is currently acting in this role, this is currently hosted by a third-party (Human Made) on behalf of the API team. For long-term usage the broker should instead be hosted on WordPress.org, alongside the existing plugin and theme repositories. This migration will take time but we remain committed to continuing to develop and support the broker.

After Merge

After merging the REST API, the team plans to continue developing the API as before. We expect that integrating the REST API into WordPress core will bring additional feedback, and we plan on incorporating this feedback through the rest of the 4.7 cycle.

During the remaining parts of this release cycle and through into the 4.8 cycle, additional work will go into other parts of the API. This includes further work and refinement on the broker authentication system, including work on WordPress.org infrastructure. Additionally, we plan to continue working on the Management API endpoints, including theme and appearance endpoints to support the Customiser team. Both of these components will be maintained as separate feature projects on GitHub until they’re ready for merge into core.

The team remains committed to supporting the API in core, and the Content API will switch from GitHub to Trac for project management and contributions. This same process occurred for the API Infrastructure in WordPress 4.4.

Reviews and Feedback

With this merge proposal, we’re looking for feedback and review of the project. In particular, we’re focussing on feedback on the security of the API and OAuth projects, and are also reaching out to specific people for reviews. (We take the security of the API seriously, and bug reports are welcomed on HackerOne at any time.) Design and accessibility reviews for the OAuth authorisation UI are also welcomed to ensure we maintain the high standards of WordPress core.

Both the REST API plugin and the OAuth plugin are available on WordPress.org, and issues can be reported to the GitHub tracker for the API and the OAuth plugin respectively. We have released a final beta (Beta 15 “International Drainage Commission”) which includes the meta and settings endpoints.

With Love from Us

As always, this is a merge proposal, and is not final until 4.7 is released. We’re eager to hear your thoughts and feedback; the comments below are a perfect place for that, or you can pop along to one of our regular meetings. We’re also always available in the #core-restapi room on Slack.

We’d like to thank every single one of our contributors, including 88 contributors to the main repository and 23 contributors to the OAuth repository. Particular thanks goes to my (@rmccue) wonderful co-lead Rachel Baker (@rachelbaker), our 2.0 release leads Daniel Bachhuber (@danielbachuber) and Joe Hoyle (@joehoyle), and our key contributors for the 4.7 cycle: Adam Silverstein (@adamsilverstein), Brian Krogsgard (@krogsgard), David Remer (@websupporter), Edwin Cromley (@chopinbach), and K. Adam White (@kadamwhite). Thanks also to the core committers helping us out through the 4.7 cycle, including Aaron D. Campbell (@aaroncampbell) and Aaron Jorbin (@aaronjorbin), and to the fantastic release lead, Helen Hou-Sandí (@helen).

Thanks also to everyone who has used the REST API, and to you for reading this. We built the REST API for you, and we hope you like it.

With love, The REST API Team

#feature-plugins, #json-api, #merge-proposals, #rest-api

REST API Team Update (4.7 week 7)

Our two (pt1) part (pt2) (split due to team travel) weekly meeting is now complete, and we have some new updates from the API team!

Team review & commentary is requested on “Open Questions on Settings” and “Open Questions on Meta”, below.

Settings

This Pull Request for adding Settings support to the API, resolving one of the few remaining major content gaps in the API, is almost ready to be merged into the rest-api feature plugin. Settings are exposed as keys on the resource `/wp/v2/settings`, e.g.:

{
    title: 'WordPress Develop',
    description: 'Just another WordPress site',
    url: 'http://src.wordpress-develop.dev',
    email: 'admin@local.dev',
    timezone: '',
    date_format: 'F j, Y',
    time_format: 'g:i a',
    start_of_week: 1,
    language: 'en_US',
    use_smilies: true,
    default_category: 1,
    default_post_format: '0',
    posts_per_page: 10
}

This resource is only accessible or modifiable if you are authenticated as a user with the `manage_options` capability.

Open Questions on Settings

Team feedback is requested on the below, and on the settings proposal in general.

  1. What do we do if a setting value in the database does not match the type used in `register_setting`?
    • This becomes a problem when there is serialized data in the `option` value. Right now we are forcing this to be a `null` value in the rest api responses. The problem with this approach is that sending that response data back to the settings endpoint will cause the deletion of that option value (as `null` in this case is essentially a delete operation).
  2. Having support for `object` recursion and `array` types in the Schema would be very nice, allowing settings that are either objects or arrays.
    • We propose that does not need to be resolved prior to merge, and may be added in a subsequent cycle.
  3. Currently, reading all options at `/wp/v2/settings` is behind a `manage_options` cap check: we need to work out if we want to publicly expose option values for some settings.
    • If we do this, we’d need a way in `register_setting` (or via some kind of `read_setting` meta cap) to work out which settings should be exposed publicly (or at specific permissions checks).
    • Site title and description are still available on the root `/wp-json/` resource index; several other settings here can be inferred from other responses or site content. We propose that this resource be left as requiring `manage_options` caps, and move establishing a mechanism in core to register a setting as public be handled in a subsequent development cycle.

Setting Registration

For your settings to be exposed in the REST API, you need to register them. WordPress includes a register_setting function which is not usually required to get/set options, but is required for API support.

To register your field, simply call register_setting and set the show_in_rest flag to true.

register_setting( 'group', 'option_name', array(
    'show_in_rest' => true,
));

register_setting Arguments

register_setting is part of WordPress core, but isn’t heavily used in most parts of core, so you may not have seen it before. Here are the options relevant to the REST API:

  • show_in_rest – Should the setting be exposed in the API? true for default behaviour, or an array of options to override defaults (see below).
  • type – This is the type of data the setting is. Can be `string`, `number`, or `boolean` (so far).
  • description – Human-readable description of what the field is for. The API exposes this in the schema.
  • sanitize_callback – Callback to sanitize the value before it’s stored in the database. See the "sanitize_{$object_type}_meta_{$meta_key}" filter.

REST API-Specific Arguments

The show_in_rest parameter defaults to false, but can be set to true to enable API support. Out of the box, the API makes some assumptions about your setting, but you can override these by setting show_in_rest to an options array instead:

register_setting( 'group', 'option_name', array(
    'show_in_rest' => array(
        'name' => 'fieldname'
    )
));

The options you can set are:

  • name – The key exposed via the REST API. For example, if your setting starts with an underscore, you may want to remove this for the API.
  • default – The default value for the setting via the REST API if it’s not set in the database.
  • schema – The JSON Schema options for the field. Exposed via OPTIONS requests to the post. This is generated automatically based on other parameters, but can be specified manually if you’d like to override it.

Settings Defaults

We’d also love feedback on ticket #38176, this adds the ability to specify a default value for an option via `register_setting()`. As you can see from the above, we are currently defining the `default` value within
the `show_in_rest` options, as there’s no way to do this in the Settings API. We’d like for there to be a way to do this in core, so `get_option()` could default to the value set in `register_setting( … [ default => ‘myvalue’ ])`
when no default is specific in `get_option`.

Meta

Meta is also ready to be merged into the plugin, and also could use review from the broader WordPress team. Similar to custom post types and taxonomies, and as discussed previously, only meta that opts-in to API support is available through the REST API.

Open Questions on Meta

  1. Right now, all meta registered through the mechanism below is assumed to be visible wherever its parent object appears. Meta values registered to a public post will be visible on that post without authentication.
    • Is it acceptable to expose registered meta fields on objects in this way, where the meta visibility will match the visibility of the parent object?
  2. As described in this comment, when you have multiple of the same value stored in a meta key, it will delete all matching values. For example, if you had a meta value [ 11, 13, 13, 13, 13 ] and attempted to remove one 13, it would result in the value [ 11 ].
  • Should multiple instances of the same value be able to be stored in a `single => false` meta field?

Meta Registration

For your meta fields to be exposed in the REST API, you need to register them. WordPress includes a register_meta function which is not usually required to get/set fields, but is required for API support.

To register your field, simply call register_meta and set the show_in_rest flag to true.

register_meta( 'post', 'field_name', array(
    'show_in_rest' => true,
));

Note: register_meta must be called separately for each meta key.

register_meta Arguments

register_meta is part of WordPress core, but isn’t heavily used in most parts of core, so you may not have seen it before. Here are the options relevant to the REST API:

  • show_in_rest – Should the field be exposed in the API? true for default behaviour, or an array of options to override defaults (see below).
  • description – Human-readable description of what the field is for. The API exposes this in the schema.
  • single – Is this field singular? WordPress allows multiple values per key on a single object, but the API needs to know whether you use this functionality.
  • sanitize_callback – Callback to sanitize the value before it’s stored in the database. See the "sanitize_{$object_type}_meta_{$meta_key}" filter.
  • auth_callback – Callback to determine whether the user can write to the field. See the "auth_post_{$post_type}_meta_{$meta_key}" filter.

REST API-Specific Arguments

The show_in_rest parameter defaults to false, but can be set to true to enable API support. Out of the box, the API makes some assumptions about your meta field, but you can override these by setting show_in_rest to an options array instead:

register_meta( 'post', 'field_name', array(
    'show_in_rest' => array(
        'name' => 'fieldname'
    )
));

The options you can set are:

  • name – The key exposed via the REST API. For example, if your meta key starts with an underscore, you may want to remove this for the API.
  • schema – The JSON Schema options for the field. Exposed via OPTIONS requests to the post. This is generated automatically based on other parameters, but can be specified manually if you’d like to override it.

Using Meta over the API

Meta is added to existing resources as a new meta field. For example, to get the meta for a single post, fetch the post at /wp/v2/posts/{id}, then look in the meta field. This is a key-value map for the registered fields.

To update a field, simply send back a new value for it.

Setting a field to null will cause the value to be removed from the database. If a field has a default value set, this essentially “resets” the value to the default (although no default will be stored in the database).

For meta fields which can hold multiple values, this field will be a JSON list. Adding or removing items from the list will create or remove the corresponding values in the database.

Other Updates

  • Add a {taxonomy}_exclude query parameter to query for resources NOT IN a particular taxonomy term_id
  • Decide against supporting full tax_query functionality via filter[tax_query]
  • Defer supporting auto-draft to a future cycle, as it did not have the expected benefits and the applicability of that state to the API is not generally understood

#4-7, #rest-api

WordPress REST API – 2.0 Beta 14

Hey folks! I’m excited to announce the release of Beta 14 of the REST API. It’s been a while since our last release, beta 14 is jam packed with general improvements, bug fixes and general refinement to polish the feature plugin before core merge.

Get it at WordPress.org or GitHub. View all changes on GitHub; here are the highlights:

  • Add support for password protected posts. There is now a protected attribute in the content and excerpt fields in post response. To view password protected posts via the API, use the password query parameter to provide the post’s password.
  • Allow returning an error from field update callbacks.
    Simply have your update_callback return a WP_Error.
  • Add relevance orderby to posts endpoint
  • Ability to order by slug, email and url on the users endpoints.
  • Add sticky parameter to the posts endpoint.
  • Update the wp-api.js client from the client-js repo.
  • Many many more

Thanks to all the contributors and special thanks to @chopinbach, @kadamwhite and @websupporter for their effort this release.

We’ll be kicking Beta 15 which will be focused on meta support for posts, terms, comments and users as well as a brand new settings endpoint. These are currently in review awaiting feedback, check out the meta pull request and the settings pull request and let us know what you think.

In addition we will be publishing a merge proposal for WordPress 4.7 this week, stay tuned!

#feature-plugins, #json-api, #rest-api