REST API Chat Summary: June 13th

This post summarizes the weekly REST API chat meeting for June 13, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Menus Endpoint

@spacedmonkey has continued work on the menus endpoint. Initial feedback was provided on the PR. The majority of the feedback has been addressed or is currently being worked on. It was decided to merge this PR soon and continue iterating in follow-up issues for the remaining feedback and decisions.

@jorgefilipecosta clarified how the Gutenberg navigation project is storing data. The first versions are using block attributes to rapidly prototype the UX. However, everything is open and using the REST API is a possibility.

@spacedmonkey brought up menu permissions. Ideally, some menu data would be publicly accessible, but that might show data that users don’t want to be public. @timothybjacobs suggested that to start, only menus that were attached to a menu location that was specifically marked as public would be exposed. A Github issue was opened for further discussion.

Upcoming Meetings

#rest-api

REST API Chat Summary: June 6, 2019

This post summarizes the weekly REST API chat meeting for June 6, 2019. (Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Supporting object & array types in register_meta()

Currently, register_meta() only supports the string, number, integer, and boolean types. #43392 proposes adding support for object and array types as well. This is particularly useful for Gutenberg blocks that save to post meta.

A preliminary patch has been added. There are a number of open questions. The main point of discussion was around how strict the REST API should be about ensuring that the current meta value is valid according to its registered schema. This is enforced in the Settings controller (which supports object and array types, but is not currently in the Meta Fields controllers.

It was decided to explore adding this validation to the Meta Fields controller. However, more feedback and comments would be appreciated. In particular from Gutenberg block authors who want to persist data to post meta.

Menus Route

@wpscholar and @spacedmonkey have been working on a menus REST API route. This is being tracked in #40878 and actively developed in the WP-API Github organization.

Scheduling a time to chat with the Gutenberg team working on Navigation would be highly valuable to explore how Gutenberg could make use of the route.

Bug Scrub

There are a number of tickets without a reply in the component. We’d like to schedule a scrub to go through this list next week. Previous secondary team meetings occurred on Tuesdays at 18:00 UTC. The next team meeting is on Thursday, June 13 at 18:00 UT where a final date and time will be decided.

#rest-api

The Block Registration API – status update

Build a WordPress.org directory for discovering blocks, and a way to seamlessly install them is one of the 9 priorities in the 2019 roadmap. This post tries to summarize work done so far and identify all the next steps required to land this project in WordPress core later this year.

It has been over two months since the Meta team published a post intended as a starting point for discussion and new ideas for the Block Directory, and a new type of plugin:

Put briefly, I’d like to propose a new type of WordPress plugin that provides blocks and nothing else: Single Block Plugins. These will be hosted in a separate Block Directory section of the Plugin Directory. They will be JavaScript-based, and each plugin will register a single Block. And they will be searchable and installable from within the Gutenberg editor itself.

@tellyworth

Currently, new Gutenberg blocks can be provided by plugins, which often register many blocks, and which are managed from outside the editor. The proposal mentioned above outlines a new type of simple block-based plugin that is intended to be seamlessly installed from within Gutenberg itself. It was later followed up with the call for design on installing blocks from within Gutenberg. There was an essential technical aspect highlighted in the post:

The WordPress.org API will provide an endpoint for searching for blocks by name and description, and return metadata similar to that of plugins. Gutenberg’s Inserter could use that endpoint to also show relevant block plugins that are available to install, with a button and process for seamless installation.

@tellyworth

This new endpoint is going to be based on the Block Registration API RFC which intends to address the server-side awareness of blocks and simplify the block discovery for the block directory project. In practical terms, it means that we are seeking for a solution where block type registration is declarative and context-agnostic. Any runtime (PHP, JS, Android, iOS or other) should be able to interpret the basics of a block type and should be able to fetch or retrieve the definitions of the context-specific implementation details.

Core Editor team reached the point where we believe that the Request for Comments is close to being finalized. However, there are still some areas where we feel that other WordPress teams could have a significant impact on the proposal.

Internationalization

The way how translations are handled inside JSON files is something novel for WordPress core. The current proposal for PHP follows the existing get_plugin_data core function which parses the plugin contents to retrieve the plugin’s metadata, and it selectively applies translations dynamically. It would be also similar for JavaScript, with the remark that we plan to implement a custom Babel plugin which would mirror PHP behavior for ESNext code. The transformation would happen during the build step to ensure that files can be statically analyzed before scripts are enqueued. You can find more details in RFC document in the Internationalization section. There is also an issue#15169 opened which describes the technical details of the proposed JavaScript implementation of the Babel plugin.

Core JS team discussed this topic at the end of the weekly meeting on Apr 2nd. We have received great feedback from @swissspidy which helped to shape the proposal. However, we still encourage other Polyglots team members to voice their opinions.

New REST API endpoints

The long term vision for the block discovery in WordPress includes:

  • Fetching the available block types through REST APIs.
  • Fetching block objects from posts through REST APIs.

The proposed implementation for the server-side awareness of block types should ensure that it stays intact with all recommendations that REST API team might have. That’s why we encourage the REST API team to get involved in the process early on. There is no immediate need to start working on new block type related API endpoints. However, it would be great to have it included on the roadmap. Ideally, they should stay as close as possible to the WordPress.org API and the final shape of the endpoint for searching for blocks.

#block-directory, #core-editor, #gutenberg, #i18n, #meta, #rest-api

REST API Meeting Agenda for March 14, 2019

The REST API weekly component chat will occur this week at March 14, 2019 18:00 UTC. If your country has recently adjusted for daylight savings time, please note that this may be a different hour than the past few months.

This week we will be continuing discussion from prior weeks around documentation needs, a canonical REST API authentication plugin, and 5.2 ticket priorities.

All agenda items are welcome, from all teams and contributors; please post them as suggestions on this document.

#agenda, #rest-api

REST API Chat Summaries: Jan 31, Feb 7

This post summarizes the weekly REST API chat meetings on January 31, 2019 and February 7, 2019. (agenda/notes, Jan 31 Slack transcript, Feb 7 Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

February 14 meeting agenda

Have a topic for discussion for today’s meeting on February 14 2019 18:00 UTC? Leave a suggested edit on the agenda document.

5.1: Dev Note needs?

  • The new rest_post_search_query filter could be called out in Other Changes.
  • changeset 44625 (update wp_die() to handle JSON contexts) could also be called out in Other Changes.

5.2 Tickets: Owners Needed

Gutenberg Widgets Endpoint

  • WordPress/gutenberg#13511: POC for a legacy widget block. @jorgefilipecosta requests input from REST API contributors.
    • Feedback provided around endpoint structure and parameter access
    • @kadamwhite proposes that user stories or use-cases for how these widgets will be consumed and displayed in the editor and rendered posts should be written for new endpoints, to inform implementation.

Authentication

  • All present agree we have a strong need for a core authentication solution. Existing plugins like the OAuth2 or JWT-Auth plugins work and are used on numerous sites, but need UX improvements and more documentation / example clients to be truly broadly applicable. OAuth2 in particular is seen as too complex / difficult to implement as a client developer.
  • Discussion on core auth since WCUS has centered on supporting basic authentication (only over SSL) (#42790), as the simplest path forward.
    • The main weakness of basic auth is that it ties all authorized applications to the user’s account name and password, so apps cannot be individually authorized or disconnected without creating new site accounts (not workable for e.g. the core mobile applications).
  • @koke thinks a JWT-based solution could be workable from the mobile applications.
  • The way CGI environments mutate authorization headers complicates any core-wide solution. A custom header may be necessary.
  • @espellcaste has volunteered to reach out to authors of existing plugin directory REST API auth solutions to get their input on what is best for core.

Upcoming Meetings

What can the REST API do for you? Join an upcoming meeting to help shape the future of this component!

#core-restapi, #meeting-notes, #rest-api

REST API Chat Summary: January 24, 2019

This post summarizes the weekly REST API chat meeting on January 24, 2019. (agenda/notes, Slack transcript). Weekly REST API component office hours are held every Thursday at 18:00 UTC in the #core-restapi room in the Make WordPress Slack.

Have a topic for discussion for the next meeting on January 31, 2019 18:00 UTC? Leave a suggested edit on next week’s agenda.

5.1 Tasks

  • One open Docs task: #45486 needs review (PHPDoc)
  • Awaiting Review ticket list needs triage & grooming but nothing currently stands out as a 5.1 priority

Tickets Awaiting Review

  • @desrosj to lead a bug scrub for this list next week, on Tuesday January 29, 18:00 UTC
  • As time permits prior to Tuesday, contributors may review that list and assign the Future Release milestone to any valid issues to show they have been triaged

5.2 Priorities

  • 6 tickets currently milestoned for 5.2. @desrosj proposes that every ticket assigned to a numbered release have an assigned owner.
  • Milestoned Tickets needing owner or review:
    • #41305 – Lazy String Evaluation: assigned to @timothybjacobs to investigate invalidating cache when locale changes within a request lifecycle
    • #39953 – Avoid updating date when modifying a “date floating” post (Related to #44975): needs owner
    • #44983 – Needs review
    • #45611 – Needs patch update to remove unused method

Gutenberg Priorities

  • content.rendered is not used by Gutenberg, and filtering the content for a large post slows down the editor load speed. @aduth suggests extending ?_fields= support to permit “deep” filtering, which the posts controller could then use to skip filtering post content where not needed, and raised the issue last week in the REST API component channel.
  • Delivering on this performance improvement within the block editor is architecturally complex, but implementing the nested _fields handling is tracked in #42094 and that ticket is now milestoned for 5.2 to lay the necessary groundwork.

Upcoming Meetings

What can the REST API do for you? Join an upcoming meeting to let the component team know!

#meeting-notes, #rest-api

REST API Meeting Agenda for January 24, 2019

The REST API team is taking inspiration from the JavaScript team and will be posting Agenda documents ahead of the meeting as Google Docs so that agenda items can be gathered from a wider and more diverse group.

All agenda items are welcome, from all teams and contributors; please post them as suggestions on this document.

This meeting will be at January 24, 2019 18:00UTC; note that this is one hour later than the last few weeks, after discussion with regular participants. As component maintainers we acknowledge there is no perfect time, but please leave a note below or feel free to contact us if this change negatively affects your ability or inclination to participate in the chat.

#agenda, #rest-api

New REST API Notice in 5.1

Edit: On January 14, 2019, the Good and Bad Practices section was added to show both correct and incorrect code examples.

Starting in WordPress 5.1, if register_rest_route() is not called on the rest_api_init action hook, a “doing it wrong” notice will be triggered. This notice is being added in an effort to encourage best practices when registering REST API endpoints.

First, let’s look at what happens when WordPress loads to set up the REST API and explore a few reasons why this pattern is beneficial.

REST API Bootstrap Process

WordPress does its best to ensure that the REST API is only loaded when a REST request is being performed. To do this, the rest_api_loaded() function is run on the parse_request action and checks for a value in the rest_route query argument. This argument is populated with a value when the Rewrite API matches a WordPress REST API URL. When a value is present, the rest_get_server() function is called to instantiate the WP_REST_Server class, store it for use across WordPress, and to run the rest_api_init action hook.

Performance

When register_rest_route() is called, it invokes rest_get_server() to retrieve the global WP_REST_Server instance created in the bootstrap process. But, if the instance has not been set up yet, it is instantiated then and the rest_api_init action hook is run. This means that every function added to the rest_api_init hook will fire at that time. This could result in a large performance hit.

For example, say register_rest_route() is called on the init action, or, just called in a theme’s functions.php file. The REST API server would be set up for every WordPress request, even those that are not actually aimed at the REST API.

Missing Endpoints

If register_rest_route() is called too early, it’s also possible that endpoints will go missing and never be registered. This happens when other plugins are not given the chance to register their rest_api_init hooks.

For example, say register_rest_route() is called directly in an mu-plugin file. This will cause the REST API to be set up before regular plugins are run, so their rest_api_init hooks will not be registered.

Good and Bad Practices

Let’s look at a few code examples and detail why they are good or bad.

Bad Practice

register_rest_route(
	'hello-world/v1',
	'/phrase',
	array(
		'method'   => 'GET',
		'callback' => 'myplugin_get_endpoint_phrase',
	)
);

In this example, register_rest_route() is called directly in a file without being attached to an action hook. This means the function will be called as soon as the file is loaded by WordPress. All of the potential issues detailed above are possible, and a _doing_it_wrong() notice will be triggered.

Good Practice

function myplugin_register_endpoints() {
	register_rest_route(
		'hello-world/v1',
		'/phrase',
		array(
			'method'   => 'GET',
			'callback' => 'myplugin_get_endpoint_phrase',
		)
	);
}
add_action( 'rest_api_init', `myplugin_register_endpoints` );

In this example, register_rest_route() is correctly placed inside of a function that is added to the rest_api_init action hook. It will only execute when the rest_api_init action hook is executed. The potential issues detailed above are avoided, and no _doing_it_wrong() notice is triggered.

Changes Required

Plugins and Themes

All plugins and themes should double check that their REST API endpoints are being registered correctly using the rest_api_init action hook. This best practice is also mentioned in the Routes and Endpoints section of the REST API Handbook. If this was a resource used when developing, chances are you won’t have to change anything!

Unit Tests

Because some unit tests require custom endpoints to exist, it is not uncommon for a test method to call register_rest_route() directly. If a test method calls the function before rest_api_init, a previously passing test method may now fail. This can be fixed in two ways.

The first way is to use rest_get_server() to create the WP_Rest_Server instance for your tests. Since rest_api_init is run within that function, this will prevent the notice. This approach can be seen in the Tests_REST_Server class. The wp_rest_server_class filter still allows you to replace the default WP_Rest_Server class with your own for testing purposes with this method.

The second way is to call do_action( 'rest_api_init' ); directly in your test method or setUp() method. This method is for scenarios where complete control is needed over the REST server setup process. This can approach can be seen in the Tests_REST_API class.

You can read more information about this change in the ticket on Trac.

#5-1, #dev-notes, #rest-api

Gutenberg and the REST API, early May

Since I last wrote two weeks ago, we’re making progress! Key achievements for Gutenberg and the REST API include:

  • Support for who=authors was added to GET wp/v2/users, making it possible to accurately query for authors. WordPress, for better or for worse, defines an author as user_level!=0. See WordPress/gutenberg#6361 for the context on why we can’t add this logic client-side (#42202 for WordPress 4.9.6).
  • Improved performance for the _fields= query parameter (e.g. GET wp/v2/pages?_fields=id,title) by ensuring WordPress core will only process the fields requested for the response. Notably, this helps us avoid running the_content when we don’t need to be (#43874 for WordPress 4.9.7).
  • Minor enhancements to reflect existing WordPress behaviors:

The “Merge Proposal: REST API” GitHub milestone represents the distance we still need to close. Slowly, steadily, we’re bridging the gap, but we could use your help. Here are some of the issues we’re still working through:

  • To ensure all necessary data is available to Gutenberg, we’ve settled upon permitting unbounded per_page=-1 REST API requests for authorized users. This landed for GET wp/v2/users (WordPress/gutenberg#6627), is in-progress for GET wp/v2/(pages|blocks) (WordPress/gutenberg#6657), and needs to be addressed for categories, tags, and custom taxonomies. We also need to patch core with this enhancement (#43998 for WordPress 4.9.7?)
  • Capabilities can’t be processed directly client-side (WordPress/gutenberg#6361), so we’ve introduced a new targetSchema concept to communicate which actions a user can perform. See it in action with wp:action-sticky (WordPress/gutenberg#6529) and wp:action-assign-author (WordPress/gutenberg#6630). There are a few other actions we will need to work out, and then we’ll need to patch core (no ticket yet).
  • @adamsilverstein is putting together an improved autosaves implementation (WordPress/gutenberg#6257) that I literally cannot wait to see complete. I’m sure he could use some help testing in the near future.
  • @flixos90 is implementing a WP_REST_Search_Controller endpoint (WordPress/gutenberg#6489) to power the link search UI.

Join us tomorrow, Thursday, May 10 at 17:00 UTC in #core-restapi office hours if you’d like to chat through any questions you have.

#gutenberg, #rest-api

REST API Meeting Summary: May 3rd

This post summarizes the REST API component team meeting from May 3rd in #core-restapi (Slack archive).

Decisions around register_meta()

This meeting revolved around finding decisions on remaining questions regarding the approach for adding subtype support to register_meta(). Please read the announcement post for the meeting for background and context.

  • It was decided to go with the fourth approach outlined in the announcement post: Every registration of a meta key is accepted, regardless of whether the same meta key already exists on the same object type in another way. Meta keys registered for an object type and subtype will take precedence over meta keys only registered for an object type (i.e. less specific). This allows us to not having to deal with conflicts. Existing calls without using object types will continue to work – while they are now fallback, chances of a conflict are generally rather low in this way. Nonetheless, a major part of these improvements will involve precise documentation and education (more on that below).
  • Easy-to-understand wrapping functions will be introduced, such as register_post_meta(), unregister_post_meta(), and the same for terms, comments and users. In those functions, we can significantly help DUX: Instead of mentioning the very abstract term “object subtype”, we can here refer to it as the more common terms “post type”, “taxonomy”, etc. respectively.
  • New filters sanitize_{$object_type}_meta_{$meta_key}_for_{$object_subtype} and auth_{$object_type}_meta_{$meta_key}_for_{$object_subtype} will be introduced for metadata sanitization and metadata capability handling for specific subtypes. If one of these two filters is being used for the given variables, the existing sanitize_{$object_type}_meta_{$meta_key} and auth_{$object_type}_meta_{$meta_key} will not be executed – again, this is because metadata registration for an object type and subtype overrules metadata registration for only an object type.

An updated patch with the above changes is in place to test the current approach, available on the ticket #38323.

Documentation & Recommendations

The existing register_meta() function has not yet seen wide use in REST API-related plugins, so the transition to this new behavior should be smooth. However, should developers continue to omit object subtypes or fail to properly prefix their meta keys, over time the chance of conflicts between or within plugins will increase—therefore precise documentation and education are necessary to make all of this work in the long run.

To aid the transition to the new behavior, it will be recommended to switch existing calls to register_meta() over to include an object subtype. Additionally, we will no longer encourage direct calls to register_meta() when a wrapping function like register_post_meta() is available—this is in line with how it is currently preferable to call get_post_meta() over get_metadata( 'post' ), for example. We will also re-emphasize the importance of prefixing meta keys to ensure uniqueness.

Related Issues

Something that we will want to consider in the future is how to deal with meta keys of the same name that are registered for exactly the same object type and the same subtype. Currently, the second call would simply override the data registered by the first call. Alternatively, we could make the second call fail, keeping the first one the “winner”. However, this is a general problem in many areas in core (think about register_post_type() for example), so this may not be the appropriate point to discuss such a big topic.

May 10th Meeting Agenda

Next week’s meeting on Thursday, May 10th 17:00 UTC will continue to focus on discussing register_meta(), particularly review whether the current patch is a viable solution. We will also try to find an answer to the last, smaller question of the original announcement post, whether comment types should actually be considered subtypes in scope of metadata registration. If there is time left afterwards, we can also discuss the state of the current directly Gutenberg-related work in the REST API – otherwise this will happen the week after.

Please review the register_meta() ticket #38323 and the latest patch there, and chime in with your thoughts on the ticket or by attending the meeting!

#rest-api, #summary