Shipping Experimental Endpoints in Core: REST API Meeting Summary June 11, 2020

This post summarizes the #core-restapi meeting for Thursday June 11, 2020. The focus of this meeting was discussing the possibility of shipping __experimental REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. routes in WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.. Read the Slack archive, account required.

Background

Part of the REST API team’s role is to help ensure that enhancements to the REST API follow Core’s established patterns and that the interface is sufficiently forward compatible.

The Problem

WordPress 5.5 is shaping up to have a number of new and improved REST API endpoints to support the latest GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ features: #40878 and #50244 for Navigation, #50321 and #48654 for the BlockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. Directory, #47620 for Block Registration, etc…

Another Gutenberg feature currently slated for 5.5 is Image Editing. This feature utilizes the REST API and existing WP_Image_Editor APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. to persist the image edits. This feature took the REST API team a bit off guard, and the team is worried about providing an adequate review of the feature in time for WordPress 5.5 given the other tickets in progress.

This feature is currently listed as a Gutenberg Experiment. The hope is to promote it from an experiment by the next Gutenberg releaseRelease A release is the distribution of the final version of an application. A software release may be either public or private and generally constitutes the initial or new generation of a new or upgraded application. A release is preceded by the distribution of alpha and then beta versions of the software..

Potential Solution

While the current server side code supporting Image Editing requires changes, it is likely that the Editor and/or Media team will be able to resolve these issues in time for BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. 1. The question then becomes if the REST API team will have the bandwidth to do a review. Additionally, will the API have had a long enough soak time to reduce the possibility of a design error necessitating a breaking change.

In the weekly #core-editor meeting, the possibility of including the required REST API endpoints in the __experimental namespace was briefly discussed. This would allow us to ship the Image Editing feature in WordPress 5.5. If breaking API changes were necessitated those could be made when the endpoint is promoted to the wp/v2 endpoint.

Note: This is distinct from the feature being marked as an “experiment” in the Gutenberg settings page. The goal is to ship this as a feature in WordPress 5.5; the Gutenberg pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party would not be required to use the feature.

Experimental APIs

Nearly all APIs in WordPress are “public” and backwards compatibility is maintained. Some APIs are explicitly marked as @access private, most notably the List Table API. Despite being private, when responsive list tables (#32395) were introduced, great lengths were made to ensure that the changes were backwards compatible with custom list tables.

Gutenberg introduced the idea of “experimental” and “unstable” APIs.

There is no support commitment for experimental and unstable APIs. They can and will be removed or changed without advance warning, including as part of a minor or patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing. release. As an external consumer, you should avoid these APIs.

An experimental API is one which is planned for eventual public availability, but is subject to further experimentation, testing, and discussion.
An unstable API is one which serves as a means to an end. It is not desired to ever be converted into a public API.

The Gutenberg Coding Guidlines

The version of Gutenberg that ships with WordPress Core utilizes a number of __experimental and __unstable APIs. But so far this has been limited to only JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. APIs.

REST API endpoints introduced in the Gutenberg plugin are currently introduced in the __experimental namespace, but are transitioned to wp/v2 before being merged to WordPress Core. Because the namespace changes, this process in itself is breaking, even if no other API changes were made.

Discussion

Editorial Note: Part of this discussion focused on the current state of the endpoint which has a number of functional and design issues. These have been flagged in the various issues and tickets and I won’t be summarizing these issues here. We would not ship an endpoint in WordPress Core, even as a proposed __experimental one, if it was broken.

@timothyblynjacobs main concern is that if there are security issues with the endpoint, or any other “fatal flaws” it being __experimental doesn’t protect us in anyway.

@spacedmonkey is concerned that once a feature is in Core, developers will start using it and it can never be removed. WordPress hasn’t shipped features in Core before they are ready and shipping an __experimental endpoint like this would break that pattern. If the feature isn’t ready, we should wait until 5.6.

@earnjam thinks that if it’s something useful, people will use it regardless of any flag/prefix applied. Then it’s a question of if we’re comfortable breaking stuff because we warned them.

To me the insistence on full backward compatibility is one of WP’s biggest assets. We’ve been drifting away from that to an extent in Gutenberg and I think it undermines a core element of what makes WordPress what it is. I think in core we had moved away from shipping private APIs for a while for this very reason, they’re going to get used whether we want them to or not. To me, that’s the whole point of doing feature plugins. It allows you to iterate and break things in the plugin, but not core. IMO the Gutenberg plugin is an experimental flag. Only the stuff that is truly ready should be brought over.

I’m ok with the experimental stuff in the Gutenberg plugin because that is opting in. I don’t know that I’m comfortable with it in core.

From @earnjam.

@jamesburke asked if the __experimental approach been a success with Gutenberg? If so, it’d be a useful pattern to implement (regardless of whether the editing endpoint is ready for this).

IMO, if you mark it as experimental, then break whatever needs to get broken. I’ve worked on production projects that heavily utilize Gutenberg experimental functionality, and it’s always nice to have early access to things. In my experience, the opportunity to use functionality in the wild and apply it to real world problems greatly outweighs the attention/effort needed to upgrade if/when something breaks on the experiment. 

From @jamesburke.

@carike points out that lot of non-technical users won’t even know that the feature exists and are not in the same position to mitigate the associated risks – particularly in use cases that are not traditional publishing, e.g. e-commerce.

@chrisvanpatten thinks that more experimentation is absolutely a good thing, but is not sure Core is the right venue for that. Gutenberg is able to play a bit fast and loose with backwards compatibility, but that has been isolated from Core to a degree. He tends to believe that __experimental APIs landing in Core has been a mistake, even as someone who has leveraged experimental JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors. APIs before.

@ajlende thinks there have been good arguments for holding off too. When he pushed the initial image editing feature, the intention was for people to start trying it out to let people use it and adjust. If Core is looking for more stable things, then he’d agree it isn’t ready.

@joemcgill mentioned that if the API is shipping in Core, even as __experimental, it absolutely will get used, so backwards compatibility must be a consideration.

@sageshilling suggested offering a quick way to turn it off the feature for sites that don’t want the image editing options. @ajlende indicated that the current code is behind a feature flag. Supporting an opt-out if the feature was included in Core would therefore likely be possible.

Next Steps

@joemcgill is fully +1 on the existence of a REST API replacement to the AJAX callbacks currently in core which is used by the image editor in the media modal and is happy to help this if it’s a priority.

@mikeschroder is completely agreed that a REST API endpoint to replace the existing AJAX is awesome.

Tickets/Issues

#49096, #44405. GB-22959, GB-21024, GB-22579.

Questions

This post is a solicitation for feedback as much as a meeting summary. Some potential questions to ponder.

  1. Have experimental APIs been a success in Gutenberg? How often are __experimental APIs transitioned to stable APIs? Is this a fast process, or do the APIs stay as __experimental for a long time? What sort of changes are discovered that necessitate breaking changes?
  2. Should we be comfortable breaking backwards compatibility because developers were explicitly warned that the feature was an experiment?

The next REST API meeting is this Thursday June 18th at 18:00 UTC.

#5-5, #gutenberg, #rest-api