REST API Chat Summary: November 14

This post summarizes the weekly REST API chat meeting for November 7, 2019. (Slack transcriptAgenda). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Authentication

A new wp-api/authentication GitHub repository was created last week to facilitate the design & development of a REST API authentication solution for WordPress Core.

We continue to be in the information gathering stage. For all interested in contributing to this effort, we will be using part of our weekly REST API office hours each Thursday at 18:00 UTC (Thursday, November 14, 2019, 01:00 PM EST) as a weekly standup to coordinate work.

We also invite you to log issues describing use cases the authentication solution should support.

Registered Block Types REST API

#47620 aims to create REST API routes to discover the list of registered block types. It is based off the Gutenberg Block Type Registration RFC. @spacedmonkey worked on a patch and is in the process of soliciting feedback from the Gutenberg team, Mobile team, and other members of the REST API team.

A particular point of concern @spacedmonkey brought up was the difficulties about handling the asset fields ( editorScript, script, editorStyle and style ). The RFC defines the fields as either absolute URLs or relative paths to the source files. However the WP_Block_Type class defines those fields as asset handles.

Further the asset URL or handle is not sufficient to make the asset functional. The list of dependencies, inline scripts, translations, and localized data are all necessary for the script to work.

@timothyblynjacobs mentioned that the RFC discusses statically discovering that information from an associated .asset.json file located “next” to the script file. @aduth mentioned that section is slightly out of date since @wordpress/scripts now outputs a .php file instead of a JSON file.

The participants discussed whether it would be better to include this additional information inline in the Block Type response, or to develop a separate wp/v2/dependencies API.

@timothyblynjacobs suggested that including this information inline would be simpler. @spacedmonkey pointed out that then we’d be including full data from a separate resource within the block type response. Elsewhere in the REST API that would be handled by creating a separate API and linking to it.

Additionally, @timothyblynjacobs pointed out that just exposing the list of dependencies isn’t sufficient. The client needs access to the entire dependency graph to ensure each dependency’s dependencies are loaded, and that all scripts are loaded in the correct order.

This all points to a dedicated REST API endpoint being a better solution. The team then discussed the potential privacy and security ramifications about retrieving this information about any registered asset.

A developer may include sensitive data in a wp_localize_script or wp_add_inline_script after registering the script with wp_register_script. Currently, this data would only be exposed when the script is enqueued, which may be protected by a current_user_can or $hook-suffix check. If the REST API allowed returning information about an arbitrary asset handle, this data may be exposed.

Additionally, a developer may conditionally registers asset based on a plugin’s settings. By allowing a user to check if a handle is registered via the REST API, it may be possible to determine the setting configuration of a plugin. This may not be desirable for security or privacy related plugins.

@kadamwhite mentioned that historically the REST API has been pretty conservative about what data is exposed. If possible, he’d like to continue along that path. Or theoretically authentication could be required for some pieces of the API since the use case seems to mostly be for editorial interfaces which would require auth anyway. @spacedmonkey also suggested a capability check.

@spacedmonkey and @timothyblynjacobs proposed limiting the assets exposed to ones registered via WP_Block_Type and all WordPress Core assets. Additional assets could be exposed via a registration flag of some kind, like show_in_rest.

Both @aduth and @youknowriad mentioned that this functionality would not just be useful for blocks. As WordPress moves more and more to JS powered interfaces, the ability to lazy load assets will become increasingly important. The problem here could be generalized to “retrieve all the information necessary to properly load a handle”.

@youknowriad opened a ticket, #48654, to continue the discussion on Trac.

Agenda for November 21

The next REST API meeting is happening in #core-restapi at Thursday, November 21 at 18:00 UTC. Agenda:

  • REST API Authentication project weekly meeting
  • Menus API discussion
  • WP Dependencies API
  • Review open tickets which should be provisionally milestoned for 5.4
  • Open floor

#meeting-notes, #rest-api

REST API Chat Summary: November 7

This post summarizes the weekly REST API chat meeting for November 7, 2019. (Slack transcript, Agenda). Please note that this meeting did not change time for daylight savings, and Weekly REST API component office hours continue to be held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack. 🙂

Authentication

The first half of the meeting discussed the newly-created wp-api/authentication GitHub repository, a follow-up to discussions at WCUS contributor day around rebooting work towards a canonical, core Authentication solution to permit the Mobile team to use the REST API instead of XMLRPC.

Our target for a merge proposal some time next year is to have an use the OAuth 2 handshake flow with dynamic client registration, which issues revocable, long-lived JWT tokens. The repo has no content so far, but we will start work by focusing on UX and the desired user-facing and technical flow rather than diving immediately into code.

@spacedmonkey, @derekherman, and others intend to drive this project over the coming months. If you who is reading this or any colleagues of yours are interested in contributing to this effort, we will be using part of our weekly REST API office hours each Thursday at 18:00 UTC (Thursday at 18:00 UTC) as a weekly standup to coordinate work.

Priorities & Goals

Priorities for Next Release

Key tickets highlighted for consideration as part of the next release cycle include, but are not limited to,

  • Improve performance in route matching #48530
  • Support registered default meta values #43941
  • Permit schema filtering #47779

If you have a ticket to highlight or propose for the next bugfix or major release, please leave it as a comment below or raise it in #core-restapi. Thank you once more, as well, to everybody who helped drive our API improvements in 5.3!

Documentation

We are behind schedule in updating the REST API handbook to cover the recent changes in WordPress 5.3. @timothyblynjacobs and @kadamwhite will be working to roll these updates out over the coming week. Handbook content is managed at github.com/wp-api/docs.

Open Floor

@timothyblynjacobs raised #44568 and #44886. Because WordPress operations are non-atomic, these race condition issues are not limited to the REST API and were determined to be out-of-scope, so #44886 was closed as wontfix.

Several bugs were raised and have been provisionally milestoned for 5.4, with the option to backport as needed once addressed.

To increase contributor awareness of REST API tickets, we discussed holding periodic component scrub meetings in the main #core channel.

Agenda for November 14

The next REST API meeting is happening shortly in #core-restapi at Thursday, November 14, 18:00 UTC. Agenda:

  • REST API Authentication project weekly meeting
  • Review open tickets which should be provisionally milestoned for 5.4
  • Open floor

#meeting-notes, #rest-api

REST API Meeting Agenda for Nov 7

The REST API weekly component chat will occur this week at November 7, 2019 18:00 UTC in the #core-restapi Slack channel.

Please note: We have not changed the UTC time for this meeting. If your country has recently adjusted for daylight savings time, this may be a different hour than the past few months.

Agenda Items:

  • Continue discussion from WordCamp US around a canonical authentication solution and begin tasking out work towards that effort
  • Discuss priorities for the 5.4 development cycle
  • Discuss needed documentation improvements
  • Open Floor

All agenda items are welcome, from all teams and contributors; please post them as comments below or let us know by joining the meeting.

#agenda, #rest-api

The REST API in WordPress 5.3

WordPress 5.3 contains a number of REST API improvements designed to make it easier and faster to work with API data from the block editor or other client applications.

Register Array & Object Metadata

As covered previously in this developer note on array & object metadata, it is now possible to use register_post_meta & register_term_meta (as well as the underlying method register_meta) to interact with complex meta values as schema-validated JSON arrays or objects using the REST API. See the linked post for more details.

Nested response filtering with _fields query parameter

This developer note on the changes to the REST API’s _fields= query parameter shows how you may now filter your REST API response objects to include only specific nested properties within the response body.

Ticket #42094

Set drafts back to “floating date” status

Once a date has been set for a draft, it was previously impossible to set the post back to showing “publish immediately” (also referred to as a “floating date,” where the post will be dated whenever it is published). As of 5.3, passing null for a date value will unset the draft date and restore this floating state.

Ticket #39953

Faster Responses

We have introduced a caching wrapper around the generation of REST resource schema objects, which initial testing has shown to yield up to a 30-40% performance increase in large API responses. If you work with expensive or large REST API queries, things should be quite a bit faster now. (Ticket #47871)

The REST API has also been improved to avoid unnecessary controller object instantiation (#45677) and to skip generation of sample permalinks when that data is not requested (#45605).

Please Note: if your team has existing performance benchmarking tooling for the REST API, please contact the component maintainers in the #core-restapi Slack channel; we very much desire to expand our metrics in this area.

Additional Changes of Note

In addition to these key enhancements, there are a number of smaller improvements to the REST API which may be of interest to developers.

  • It is no longer possible to DELETE a Revision resource using the REST API, as this behavior could break a post’s audit trail. Ticket #43709
  • The /search endpoint will now correctly embed the full original body of each matched resource when passing the _embed query parameter. Ticket #47684
  • rest_do_request and rest_ensure_request now accept a string API path, so it is possible to instantiate a request in PHP using nothing more than the desired endpoint string, e.g.
    rest_do_request( '/wp/v2/posts' );
    Ticket #40614
  • Creating or updating a Terms resource via the REST API now returns the updated object using the “edit” context. Ticket #41411
  • It is now possible to edit a posted comment through the REST API when authenticating the request as a user with the moderate_comments capability. Previously a full editor- or admin-level role was needed. Ticket #47024
  • rest_get_avatar_urls now receives the entire User or Comment object, not just the object’s email address. Ticket #40030

Welcoming Timothy Jacobs as a REST API component maintainer

Last but not least, many of you have no doubt seen @timothyblynjacobs active in trac, slack, and community events. Timothy has driven much of the momentum that resulted in the above improvements, and I’m excited to (belatedly) announce that he has joined the REST API team as an official component maintainer. Thank you very much for your energy and dedication!

Thank you also to every other person who contributed to API changes this cycle; it’s the best version of the REST API yet, and we couldn’t have done it without the dozens of contributors who helped create, review and land these patches.

We’ve got some ambitious ideas about how we can make the REST API even better in 5.4. Interested in helping out, with code, docs, or triage? Join us for weekly office hours, every week on Thursdays at 1800 UTC!

#5-3, #dev-notes, #rest-api

Filtering nested REST response _fields in WP 5.3

WordPress 4.9.8 introduced the ability to limit the fields included in the JSON objects returned from the REST API, for example specifying

/wp/v2/posts?_fields=id,title,author

to return a list of posts with only id, title & author fields in situations where we don’t need all of the data contained in other fields like content or media (see #38131). Since 4.9.8 we’ve made further improvements to skip computing fields we did not explicitly request when _fields is present, saving time on the server in addition to slimming down the JSON response object.

In WordPress 5.3 we are adding the ability to filter by nested fields. Previously we could only request top-level properties like content or meta, which would return the full content object (with raw and rendered properties when using an edit context) or the object containing all meta values. We can now specify a nested path such as content.raw and the REST API will skip computing the rendered content, a useful performance boost for applications like Gutenberg which only require that underlying raw post content.

Now that we can register complex array or object meta, we may similarly ask for only a few of many registered meta fields, or certain properties within a complex object, using a query such as this:

?_fields=meta.meta-key-1,meta.meta-key-2,meta.meta-key-3.nested-prop

(Note that this specific meta example depends on bugfix #48266, which will ship as part of RC1.)

Thank you @timothyblynjacobs, @dlh, @danielbachhuber, and @rmccue for assisting with the development of this useful feature!

#5-3, #dev-notes, #rest-api

WP 5.3 Supports Object and Array Meta Types in the REST API

With WordPress 5.3, the register_meta functions (including register_post_meta) now support the 'object' and 'array' data types. Previously, the recommended solution to create complex meta-based Gutenberg blocks was to JSON encode the block’s complex attribute value, and pass that string to the API. The REST API now natively supports those complex metadata types. This allows leveraging the REST API to perform schema-based validation, and should additionally simplify client code which interacts with these complex values through the REST API.

Importantly, these data types follow the JSON specification, not the PHP definition. For comparison, that means a JSON object type is equivalent to an associative array in PHP. The JSON array type is a numerically indexed array.

When registering a complex meta field, it is almost always required to also specify a JSON schema describing the expected structure. This can be done by switching show_in_rest from a simple true value, to an array that specifies the desired schema under the schema array key.

Object Example

The following code sample registers a post meta field called “release” that accepts the given JSON data.

{
  "meta": {
    "release": {
      "version": "5.2",
      "artist": "Jaco"
    }
  }
}
register_post_meta(
     'post',
     'release',
     array(
         'single'       => true,
         'type'         => 'object',
         'show_in_rest' => array(
             'schema' => array(
                 'type'       => 'object',
                 'properties' => array(
                     'version' => array(
                         'type' => 'string',
                     ),
                     'artist'  => array(
                         'type' => 'string',
                     ),
                 ),
             ),
         ),
     )
 );

By default, only properties that are explicitly specified in the schema will be allowed. The additionalProperties keyword can be used to to change this behavior. additionalProperties should be another JSON schema that is used to validate any unknown object members. For instance, to enforce that any additional properties are numbers, the following code can be used.

{
   "meta": {
     "release": {
       "version": "5.2",
       "artist": "Jaco",
       "unknown_field": 5.3
     }
   }
 }
register_post_meta(
     'post',
     'version',
     array(
         'single'       => true,
         'type'         => 'object',
         'show_in_rest' => array(
             'schema' => array(
                 'type'       => 'object',
                 'properties' => array(
                     'version' => array(
                         'type' => 'string',
                     ),
                     'artist'  => array(
                         'type' => 'string',
                     ),
                 ),
                 'additionalProperties' => array(
                     'type' => 'number',
                 ),
             ),
         ),
     )
 );

Additionally, additionalProperties can be set to true to allow unknown properties of any format, but this is not recommended.

Array Example

The following code sample registers a post meta field called “projects” that contains a list of project names that accepts the given JSON data.

{
   "meta": {
     "projects": [
       "WordPress",
       "BuddyPress"
     ]
   }
 }
register_post_meta(
     'post',
     'projects',
     array(
         'single'       => true,
         'type'         => 'array',
         'show_in_rest' => array(
             'schema' => array(
                 'type'  => 'array',
                 'items' => array(
                     'type' => 'string',
                 ),
             ),
         ),
     )
 );

The “items” keyword is used to define the JSON schema to validate each array member against. It can be a simple type like “string” or a complex type like “object”.

For instance, to accept the given JSON data, the following meta registration would be used.

{
   "meta": {
     "projects": [
       {
         "name": "WordPress",
         "website": "https://wordpress.org"
       },
       {
         "name": "BuddyPress",
         "website": "https://buddypress.org"
       }
     ]
   }
 }
register_post_meta(
     'post',
     'projects',
     array(
         'single'       => true,
         'type'         => 'array',
         'show_in_rest' => array(
             'schema' => array(
                 'items' => array(
                     'type'       => 'object',
                     'properties' => array(
                         'name'    => array(
                             'type' => 'string',
                         ),
                         'website' => array(
                             'type'   => 'string',
                             'format' => 'uri',
                         ),
                     ),
                 ),
             ),
         ),
     )
 );

Non-Single Metadata

Non-single meta fields have an array of values per post, instead of one value per post. Each of those values is stored in a separate row in the postmeta table.

The array and object data types can be used with non-single meta fields as well. For example, if the “release” meta key from earlier had single set to false, the following JSON data could be accepted.

{
   "meta": {
     "release": [
       {
         "version": "5.2",
         "artist": "Jaco"
       },
       {
         "version": "5.1",
         "artist": "Betty"
       }
     ]
   }
 }

This would result in two postmeta database rows. The first containing { "version": "5.2", "artist": "Jaco" } and the second containing { "version": "5.1", "artist": "Betty" }.

Similarly, the following data would be accepted for the “projects” example if it had set single to false.

{
   "meta": {
     "projects": [
       [
         "WordPress",
         "BuddyPress"
       ],
       [
         "bbPress"
       ]
     ]
   }
 }

This would result in two postmeta database rows. The first containing [ "WordPress", "BuddyPress" ] and the second containing [ "bbPress" ].

Invalid Stored Values

If the existing value for a meta field does not validate against the registered type and schema, the value for that meta field will be returned as null. This is a change in 5.3, where previously only the meta type was validated.

#5-3, #dev-notes, #rest-api

Dev Chat Summary: August 14

After the close of our every-two-weeks new contributor chat, the weekly core chat started at 2000 UTC, give or take a few minutes. (backscroll)

Announcements

Next Minor: 5.2.3

Next Major: 5.3

  • All but two focus lead type people are settled. An update post is upcoming (and will be shared by the end of the week regardless of whether those final two are settled or not).

Open Floor

To Do List from this Chat

  • First 5.2.3 bug scrub Thursday, August 15 @ 1700 UTC
  • If you want to help with the 5.2.3 minor release and weren’t mentioned above, you can indicate your interest in the comments of this post.

#summary #5-2-3 #5-3 #rest-api #auto-update

Rest API Chat Summary: June 27

This post summarizes the weekly REST API chat meeting for June 27, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Ticket Review

#43691 is close to being ready for commit. @timothybjacobs raised the question of how strict WordPress should be about enforcing the use of the 204 status code. @kadamwhite agreed that core shouldn’t be that stringent.

#43392 has received some more feedback. @timothybjacobs is working on a refreshed patch. @spacedmonkey wanted to know how the ticket would impact #43941. @kadamwhite didn’t think they’d conflict and whichever ticket landed first would require the other to be adjusted.

#47443 came up in a recent REST API bug scrub. It touches post type capabilities which can be tricky. Feedback from developers with post type capabilities experience would be very helpful.

Menus Endpoint

@spacedmonkey has continued work on the Menus Endpoint. A discussion ensued about how to get better feedback on open PRs. Johnny is planning on writing up some basic test instructions. Subsequent PRs will hopefully be smaller and easier to test in the future.

The Menus Location endpoint was discussed. In particular, what endpoint should be responsible for assigning a menu to a menu location. @spacedmonkey‘s gut reaction is that menu-locations are like categories. You assign a category on the post endpoint, so a menu location should assigned on the menu. @kadamwhite and @timothybjacobs had similar initial thoughts.

@kadamwhite brought up that it’s a bit of a flip from the existing UI model, where you start with a location and then add a menu to it.

@timothybjacobs suggested modeling it after the existing posts controller, at least for the first iteration.

Upcoming Meetings

#rest-api

REST API Chat Summary: June 13th

This post summarizes the weekly REST API chat meeting for June 13, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Menus Endpoint

@spacedmonkey has continued work on the menus endpoint. Initial feedback was provided on the PR. The majority of the feedback has been addressed or is currently being worked on. It was decided to merge this PR soon and continue iterating in follow-up issues for the remaining feedback and decisions.

@jorgefilipecosta clarified how the Gutenberg navigation project is storing data. The first versions are using block attributes to rapidly prototype the UX. However, everything is open and using the REST API is a possibility.

@spacedmonkey brought up menu permissions. Ideally, some menu data would be publicly accessible, but that might show data that users don’t want to be public. @timothybjacobs suggested that to start, only menus that were attached to a menu location that was specifically marked as public would be exposed. A Github issue was opened for further discussion.

Upcoming Meetings

#rest-api

REST API Chat Summary: June 6, 2019

This post summarizes the weekly REST API chat meeting for June 6, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Supporting object & array types in register_meta()

Currently, register_meta() only supports the string, number, integer, and boolean types. #43392 proposes adding support for object and array types as well. This is particularly useful for Gutenberg blocks that save to post meta.

A preliminary patch has been added. There are a number of open questions. The main point of discussion was around how strict the REST API should be about ensuring that the current meta value is valid according to its registered schema. This is enforced in the Settings controller (which supports object and array types, but is not currently in the Meta Fields controllers.

It was decided to explore adding this validation to the Meta Fields controller. However, more feedback and comments would be appreciated. In particular from Gutenberg block authors who want to persist data to post meta.

Menus Route

@wpscholar and @spacedmonkey have been working on a menus REST API route. This is being tracked in #40878 and actively developed in the WP-API Github organization.

Scheduling a time to chat with the Gutenberg team working on Navigation would be highly valuable to explore how Gutenberg could make use of the route.

Bug Scrub

There are a number of tickets without a reply in the component. We’d like to schedule a scrub to go through this list next week. Previous secondary team meetings occurred on Tuesdays at 18:00 UTC. The next team meeting is on Thursday, June 13 at 18:00 UT where a final date and time will be decided.

#rest-api