The REST API in WordPress 5.3

WordPress 5.3 contains a number of REST API improvements designed to make it easier and faster to work with API data from the block editor or other client applications.

Register Array & Object Metadata

As covered previously in this developer note on array & object metadata, it is now possible to use register_post_meta & register_term_meta (as well as the underlying method register_meta) to interact with complex meta values as schema-validated JSON arrays or objects using the REST API. See the linked post for more details.

Nested response filtering with _fields query parameter

This developer note on the changes to the REST API’s _fields= query parameter shows how you may now filter your REST API response objects to include only specific nested properties within the response body.

Ticket #42094

Set drafts back to “floating date” status

Once a date has been set for a draft, it was previously impossible to set the post back to showing “publish immediately” (also referred to as a “floating date,” where the post will be dated whenever it is published). As of 5.3, passing null for a date value will unset the draft date and restore this floating state.

Ticket #39953

Faster Responses

We have introduced a caching wrapper around the generation of REST resource schema objects, which initial testing has shown to yield up to a 30-40% performance increase in large API responses. If you work with expensive or large REST API queries, things should be quite a bit faster now. (Ticket #47871)

The REST API has also been improved to avoid unnecessary controller object instantiation (#45677) and to skip generation of sample permalinks when that data is not requested (#45605).

Please Note: if your team has existing performance benchmarking tooling for the REST API, please contact the component maintainers in the #core-restapi Slack channel; we very much desire to expand our metrics in this area.

Additional Changes of Note

In addition to these key enhancements, there are a number of smaller improvements to the REST API which may be of interest to developers.

  • It is no longer possible to DELETE a Revision resource using the REST API, as this behavior could break a post’s audit trail. Ticket #43709
  • The /search endpoint will now correctly embed the full original body of each matched resource when passing the _embed query parameter. Ticket #47684
  • rest_do_request and rest_ensure_request now accept a string API path, so it is possible to instantiate a request in PHP using nothing more than the desired endpoint string, e.g.
    rest_do_request( '/wp/v2/posts' );
    Ticket #40614
  • Creating or updating a Terms resource via the REST API now returns the updated object using the “edit” context. Ticket #41411
  • It is now possible to edit a posted comment through the REST API when authenticating the request as a user with the moderate_comments capability. Previously a full editor- or admin-level role was needed. Ticket #47024
  • rest_get_avatar_urls now receives the entire User or Comment object, not just the object’s email address. Ticket #40030

Welcoming Timothy Jacobs as a REST API component maintainer

Last but not least, many of you have no doubt seen @timothyblynjacobs active in trac, slack, and community events. Timothy has driven much of the momentum that resulted in the above improvements, and I’m excited to (belatedly) announce that he has joined the REST API team as an official component maintainer. Thank you very much for your energy and dedication!

Thank you also to every other person who contributed to API changes this cycle; it’s the best version of the REST API yet, and we couldn’t have done it without the dozens of contributors who helped create, review and land these patches.

We’ve got some ambitious ideas about how we can make the REST API even better in 5.4. Interested in helping out, with code, docs, or triage? Join us for weekly office hours, every week on Thursdays at 1800 UTC!

#5-3, #dev-notes, #rest-api

Filtering nested REST response _fields in WP 5.3

WordPress 4.9.8 introduced the ability to limit the fields included in the JSON objects returned from the REST API, for example specifying

/wp/v2/posts?_fields=id,title,author

to return a list of posts with only id, title & author fields in situations where we don’t need all of the data contained in other fields like content or media (see #38131). Since 4.9.8 we’ve made further improvements to skip computing fields we did not explicitly request when _fields is present, saving time on the server in addition to slimming down the JSON response object.

In WordPress 5.3 we are adding the ability to filter by nested fields. Previously we could only request top-level properties like content or meta, which would return the full content object (with raw and rendered properties when using an edit context) or the object containing all meta values. We can now specify a nested path such as content.raw and the REST API will skip computing the rendered content, a useful performance boost for applications like Gutenberg which only require that underlying raw post content.

Now that we can register complex array or object meta, we may similarly ask for only a few of many registered meta fields, or certain properties within a complex object, using a query such as this:

?_fields=meta.meta-key-1,meta.meta-key-2,meta.meta-key-3.nested-prop

(Note that this specific meta example depends on bugfix #48266, which will ship as part of RC1.)

Thank you @timothyblynjacobs, @dlh, @danielbachhuber, and @rmccue for assisting with the development of this useful feature!

#5-3, #dev-notes, #rest-api

WP 5.3 Supports Object and Array Meta Types in the REST API

With WordPress 5.3, the register_meta functions (including register_post_meta) now support the 'object' and 'array' data types. Previously, the recommended solution to create complex meta-based Gutenberg blocks was to JSON encode the block’s complex attribute value, and pass that string to the API. The REST API now natively supports those complex metadata types. This allows leveraging the REST API to perform schema-based validation, and should additionally simplify client code which interacts with these complex values through the REST API.

Importantly, these data types follow the JSON specification, not the PHP definition. For comparison, that means a JSON object type is equivalent to an associative array in PHP. The JSON array type is a numerically indexed array.

When registering a complex meta field, it is almost always required to also specify a JSON schema describing the expected structure. This can be done by switching show_in_rest from a simple true value, to an array that specifies the desired schema under the schema array key.

Object Example

The following code sample registers a post meta field called “release” that accepts the given JSON data.

{
  "meta": {
    "release": {
      "version": "5.2",
      "artist": "Jaco"
    }
  }
}
register_post_meta(
     'post',
     'release',
     array(
         'single'       => true,
         'type'         => 'object',
         'show_in_rest' => array(
             'schema' => array(
                 'type'       => 'object',
                 'properties' => array(
                     'version' => array(
                         'type' => 'string',
                     ),
                     'artist'  => array(
                         'type' => 'string',
                     ),
                 ),
             ),
         ),
     )
 );

By default, only properties that are explicitly specified in the schema will be allowed. The additionalProperties keyword can be used to to change this behavior. additionalProperties should be another JSON schema that is used to validate any unknown object members. For instance, to enforce that any additional properties are numbers, the following code can be used.

{
   "meta": {
     "release": {
       "version": "5.2",
       "artist": "Jaco",
       "unknown_field": 5.3
     }
   }
 }
register_post_meta(
     'post',
     'version',
     array(
         'single'       => true,
         'type'         => 'object',
         'show_in_rest' => array(
             'schema' => array(
                 'type'       => 'object',
                 'properties' => array(
                     'version' => array(
                         'type' => 'string',
                     ),
                     'artist'  => array(
                         'type' => 'string',
                     ),
                 ),
                 'additionalProperties' => array(
                     'type' => 'number',
                 ),
             ),
         ),
     )
 );

Additionally, additionalProperties can be set to true to allow unknown properties of any format, but this is not recommended.

Array Example

The following code sample registers a post meta field called “projects” that contains a list of project names that accepts the given JSON data.

{
   "meta": {
     "projects": [
       "WordPress",
       "BuddyPress"
     ]
   }
 }
register_post_meta(
     'post',
     'projects',
     array(
         'single'       => true,
         'type'         => 'array',
         'show_in_rest' => array(
             'schema' => array(
                 'type'  => 'array',
                 'items' => array(
                     'type' => 'string',
                 ),
             ),
         ),
     )
 );

The “items” keyword is used to define the JSON schema to validate each array member against. It can be a simple type like “string” or a complex type like “object”.

For instance, to accept the given JSON data, the following meta registration would be used.

{
   "meta": {
     "projects": [
       {
         "name": "WordPress",
         "website": "https://wordpress.org"
       },
       {
         "name": "BuddyPress",
         "website": "https://buddypress.org"
       }
     ]
   }
 }
register_post_meta(
     'post',
     'projects',
     array(
         'single'       => true,
         'type'         => 'array',
         'show_in_rest' => array(
             'schema' => array(
                 'items' => array(
                     'type'       => 'object',
                     'properties' => array(
                         'name'    => array(
                             'type' => 'string',
                         ),
                         'website' => array(
                             'type'   => 'string',
                             'format' => 'uri',
                         ),
                     ),
                 ),
             ),
         ),
     )
 );

Non-Single Metadata

Non-single meta fields have an array of values per post, instead of one value per post. Each of those values is stored in a separate row in the postmeta table.

The array and object data types can be used with non-single meta fields as well. For example, if the “release” meta key from earlier had single set to false, the following JSON data could be accepted.

{
   "meta": {
     "release": [
       {
         "version": "5.2",
         "artist": "Jaco"
       },
       {
         "version": "5.1",
         "artist": "Betty"
       }
     ]
   }
 }

This would result in two postmeta database rows. The first containing { "version": "5.2", "artist": "Jaco" } and the second containing { "version": "5.1", "artist": "Betty" }.

Similarly, the following data would be accepted for the “projects” example if it had set single to false.

{
   "meta": {
     "projects": [
       [
         "WordPress",
         "BuddyPress"
       ],
       [
         "bbPress"
       ]
     ]
   }
 }

This would result in two postmeta database rows. The first containing [ "WordPress", "BuddyPress" ] and the second containing [ "bbPress" ].

Invalid Stored Values

If the existing value for a meta field does not validate against the registered type and schema, the value for that meta field will be returned as null. This is a change in 5.3, where previously only the meta type was validated.

#5-3, #dev-notes, #rest-api

Dev Chat Summary: August 14

After the close of our every-two-weeks new contributor chat, the weekly core chat started at 2000 UTC, give or take a few minutes. (backscroll)

Announcements

Next Minor: 5.2.3

Next Major: 5.3

  • All but two focus lead type people are settled. An update post is upcoming (and will be shared by the end of the week regardless of whether those final two are settled or not).

Open Floor

To Do List from this Chat

  • First 5.2.3 bug scrub Thursday, August 15 @ 1700 UTC
  • If you want to help with the 5.2.3 minor release and weren’t mentioned above, you can indicate your interest in the comments of this post.

#summary #5-2-3 #5-3 #rest-api #auto-update

Rest API Chat Summary: June 27

This post summarizes the weekly REST API chat meeting for June 27, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Ticket Review

#43691 is close to being ready for commit. @timothybjacobs raised the question of how strict WordPress should be about enforcing the use of the 204 status code. @kadamwhite agreed that core shouldn’t be that stringent.

#43392 has received some more feedback. @timothybjacobs is working on a refreshed patch. @spacedmonkey wanted to know how the ticket would impact #43941. @kadamwhite didn’t think they’d conflict and whichever ticket landed first would require the other to be adjusted.

#47443 came up in a recent REST API bug scrub. It touches post type capabilities which can be tricky. Feedback from developers with post type capabilities experience would be very helpful.

Menus Endpoint

@spacedmonkey has continued work on the Menus Endpoint. A discussion ensued about how to get better feedback on open PRs. Johnny is planning on writing up some basic test instructions. Subsequent PRs will hopefully be smaller and easier to test in the future.

The Menus Location endpoint was discussed. In particular, what endpoint should be responsible for assigning a menu to a menu location. @spacedmonkey‘s gut reaction is that menu-locations are like categories. You assign a category on the post endpoint, so a menu location should assigned on the menu. @kadamwhite and @timothybjacobs had similar initial thoughts.

@kadamwhite brought up that it’s a bit of a flip from the existing UI model, where you start with a location and then add a menu to it.

@timothybjacobs suggested modeling it after the existing posts controller, at least for the first iteration.

Upcoming Meetings

#rest-api

REST API Chat Summary: June 13th

This post summarizes the weekly REST API chat meeting for June 13, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Menus Endpoint

@spacedmonkey has continued work on the menus endpoint. Initial feedback was provided on the PR. The majority of the feedback has been addressed or is currently being worked on. It was decided to merge this PR soon and continue iterating in follow-up issues for the remaining feedback and decisions.

@jorgefilipecosta clarified how the Gutenberg navigation project is storing data. The first versions are using block attributes to rapidly prototype the UX. However, everything is open and using the REST API is a possibility.

@spacedmonkey brought up menu permissions. Ideally, some menu data would be publicly accessible, but that might show data that users don’t want to be public. @timothybjacobs suggested that to start, only menus that were attached to a menu location that was specifically marked as public would be exposed. A Github issue was opened for further discussion.

Upcoming Meetings

#rest-api

REST API Chat Summary: June 6, 2019

This post summarizes the weekly REST API chat meeting for June 6, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Supporting object & array types in register_meta()

Currently, register_meta() only supports the string, number, integer, and boolean types. #43392 proposes adding support for object and array types as well. This is particularly useful for Gutenberg blocks that save to post meta.

A preliminary patch has been added. There are a number of open questions. The main point of discussion was around how strict the REST API should be about ensuring that the current meta value is valid according to its registered schema. This is enforced in the Settings controller (which supports object and array types, but is not currently in the Meta Fields controllers.

It was decided to explore adding this validation to the Meta Fields controller. However, more feedback and comments would be appreciated. In particular from Gutenberg block authors who want to persist data to post meta.

Menus Route

@wpscholar and @spacedmonkey have been working on a menus REST API route. This is being tracked in #40878 and actively developed in the WP-API Github organization.

Scheduling a time to chat with the Gutenberg team working on Navigation would be highly valuable to explore how Gutenberg could make use of the route.

Bug Scrub

There are a number of tickets without a reply in the component. We’d like to schedule a scrub to go through this list next week. Previous secondary team meetings occurred on Tuesdays at 18:00 UTC. The next team meeting is on Thursday, June 13 at 18:00 UT where a final date and time will be decided.

#rest-api

The Block Registration API – status update

Build a WordPress.org directory for discovering blocks, and a way to seamlessly install them is one of the 9 priorities in the 2019 roadmap. This post tries to summarize work done so far and identify all the next steps required to land this project in WordPress core later this year.

It has been over two months since the Meta team published a post intended as a starting point for discussion and new ideas for the Block Directory, and a new type of plugin:

Put briefly, I’d like to propose a new type of WordPress plugin that provides blocks and nothing else: Single Block Plugins. These will be hosted in a separate Block Directory section of the Plugin Directory. They will be JavaScript-based, and each plugin will register a single Block. And they will be searchable and installable from within the Gutenberg editor itself.

@tellyworth

Currently, new Gutenberg blocks can be provided by plugins, which often register many blocks, and which are managed from outside the editor. The proposal mentioned above outlines a new type of simple block-based plugin that is intended to be seamlessly installed from within Gutenberg itself. It was later followed up with the call for design on installing blocks from within Gutenberg. There was an essential technical aspect highlighted in the post:

The WordPress.org API will provide an endpoint for searching for blocks by name and description, and return metadata similar to that of plugins. Gutenberg’s Inserter could use that endpoint to also show relevant block plugins that are available to install, with a button and process for seamless installation.

@tellyworth

This new endpoint is going to be based on the Block Registration API RFC which intends to address the server-side awareness of blocks and simplify the block discovery for the block directory project. In practical terms, it means that we are seeking for a solution where block type registration is declarative and context-agnostic. Any runtime (PHP, JS, Android, iOS or other) should be able to interpret the basics of a block type and should be able to fetch or retrieve the definitions of the context-specific implementation details.

Core Editor team reached the point where we believe that the Request for Comments is close to being finalized. However, there are still some areas where we feel that other WordPress teams could have a significant impact on the proposal.

Internationalization

The way how translations are handled inside JSON files is something novel for WordPress core. The current proposal for PHP follows the existing get_plugin_data core function which parses the plugin contents to retrieve the plugin’s metadata, and it selectively applies translations dynamically. It would be also similar for JavaScript, with the remark that we plan to implement a custom Babel plugin which would mirror PHP behavior for ESNext code. The transformation would happen during the build step to ensure that files can be statically analyzed before scripts are enqueued. You can find more details in RFC document in the Internationalization section. There is also an issue#15169 opened which describes the technical details of the proposed JavaScript implementation of the Babel plugin.

Core JS team discussed this topic at the end of the weekly meeting on Apr 2nd. We have received great feedback from @swissspidy which helped to shape the proposal. However, we still encourage other Polyglots team members to voice their opinions.

New REST API endpoints

The long term vision for the block discovery in WordPress includes:

  • Fetching the available block types through REST APIs.
  • Fetching block objects from posts through REST APIs.

The proposed implementation for the server-side awareness of block types should ensure that it stays intact with all recommendations that REST API team might have. That’s why we encourage the REST API team to get involved in the process early on. There is no immediate need to start working on new block type related API endpoints. However, it would be great to have it included on the roadmap. Ideally, they should stay as close as possible to the WordPress.org API and the final shape of the endpoint for searching for blocks.

#block-directory, #core-editor, #gutenberg, #i18n, #meta, #rest-api

REST API Meeting Agenda for March 14, 2019

The REST API weekly component chat will occur this week at March 14, 2019 18:00 UTC. If your country has recently adjusted for daylight savings time, please note that this may be a different hour than the past few months.

This week we will be continuing discussion from prior weeks around documentation needs, a canonical REST API authentication plugin, and 5.2 ticket priorities.

All agenda items are welcome, from all teams and contributors; please post them as suggestions on this document.

#agenda, #rest-api

REST API Chat Summaries: Jan 31, Feb 7

This post summarizes the weekly REST API chat meetings on January 31, 2019 and February 7, 2019. (agenda/notes, Jan 31 Slack transcript, Feb 7 Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

February 14 meeting agenda

Have a topic for discussion for today’s meeting on February 14 2019 18:00 UTC? Leave a suggested edit on the agenda document.

5.1: Dev Note needs?

  • The new rest_post_search_query filter could be called out in Other Changes.
  • changeset 44625 (update wp_die() to handle JSON contexts) could also be called out in Other Changes.

5.2 Tickets: Owners Needed

Gutenberg Widgets Endpoint

  • WordPress/gutenberg#13511: POC for a legacy widget block. @jorgefilipecosta requests input from REST API contributors.
    • Feedback provided around endpoint structure and parameter access
    • @kadamwhite proposes that user stories or use-cases for how these widgets will be consumed and displayed in the editor and rendered posts should be written for new endpoints, to inform implementation.

Authentication

  • All present agree we have a strong need for a core authentication solution. Existing plugins like the OAuth2 or JWT-Auth plugins work and are used on numerous sites, but need UX improvements and more documentation / example clients to be truly broadly applicable. OAuth2 in particular is seen as too complex / difficult to implement as a client developer.
  • Discussion on core auth since WCUS has centered on supporting basic authentication (only over SSL) (#42790), as the simplest path forward.
    • The main weakness of basic auth is that it ties all authorized applications to the user’s account name and password, so apps cannot be individually authorized or disconnected without creating new site accounts (not workable for e.g. the core mobile applications).
  • @koke thinks a JWT-based solution could be workable from the mobile applications.
  • The way CGI environments mutate authorization headers complicates any core-wide solution. A custom header may be necessary.
  • @espellcaste has volunteered to reach out to authors of existing plugin directory REST API auth solutions to get their input on what is best for core.

Upcoming Meetings

What can the REST API do for you? Join an upcoming meeting to help shape the future of this component!

#core-restapi, #meeting-notes, #rest-api