Gutenberg and the REST API, early May

Since I last wrote two weeks ago, we’re making progress! Key achievements for Gutenberg and the REST API include:

  • Support for who=authors was added to GET wp/v2/users, making it possible to accurately query for authors. WordPress, for better or for worse, defines an author as user_level!=0. See WordPress/gutenberg#6361 for the context on why we can’t add this logic client-side (#42202 for WordPress 4.9.6).
  • Improved performance for the _fields= query parameter (e.g. GET wp/v2/pages?_fields=id,title) by ensuring WordPress core will only process the fields requested for the response. Notably, this helps us avoid running the_content when we don’t need to be (#43874 for WordPress 4.9.7).
  • Minor enhancements to reflect existing WordPress behaviors:

The “Merge Proposal: REST API” GitHub milestone represents the distance we still need to close. Slowly, steadily, we’re bridging the gap, but we could use your help. Here are some of the issues we’re still working through:

  • To ensure all necessary data is available to Gutenberg, we’ve settled upon permitting unbounded per_page=-1 REST API requests for authorized users. This landed for GET wp/v2/users (WordPress/gutenberg#6627), is in-progress for GET wp/v2/(pages|blocks) (WordPress/gutenberg#6657), and needs to be addressed for categories, tags, and custom taxonomies. We also need to patch core with this enhancement (#43998 for WordPress 4.9.7?)
  • Capabilities can’t be processed directly client-side (WordPress/gutenberg#6361), so we’ve introduced a new targetSchema concept to communicate which actions a user can perform. See it in action with wp:action-sticky (WordPress/gutenberg#6529) and wp:action-assign-author (WordPress/gutenberg#6630). There are a few other actions we will need to work out, and then we’ll need to patch core (no ticket yet).
  • @adamsilverstein is putting together an improved autosaves implementation (WordPress/gutenberg#6257) that I literally cannot wait to see complete. I’m sure he could use some help testing in the near future.
  • @flixos90 is implementing a WP_REST_Search_Controller endpoint (WordPress/gutenberg#6489) to power the link search UI.

Join us tomorrow, Thursday, May 10 at 17:00 UTC in #core-restapi office hours if you’d like to chat through any questions you have.

#gutenberg, #rest-api

REST API Meeting Summary: May 3rd

This post summarizes the REST API component team meeting from May 3rd in #core-restapi (Slack archive).

Decisions around register_meta()

This meeting revolved around finding decisions on remaining questions regarding the approach for adding subtype support to register_meta(). Please read the announcement post for the meeting for background and context.

  • It was decided to go with the fourth approach outlined in the announcement post: Every registration of a meta key is accepted, regardless of whether the same meta key already exists on the same object type in another way. Meta keys registered for an object type and subtype will take precedence over meta keys only registered for an object type (i.e. less specific). This allows us to not having to deal with conflicts. Existing calls without using object types will continue to work – while they are now fallback, chances of a conflict are generally rather low in this way. Nonetheless, a major part of these improvements will involve precise documentation and education (more on that below).
  • Easy-to-understand wrapping functions will be introduced, such as register_post_meta(), unregister_post_meta(), and the same for terms, comments and users. In those functions, we can significantly help DUX: Instead of mentioning the very abstract term “object subtype”, we can here refer to it as the more common terms “post type”, “taxonomy”, etc. respectively.
  • New filters sanitize_{$object_type}_meta_{$meta_key}_for_{$object_subtype} and auth_{$object_type}_meta_{$meta_key}_for_{$object_subtype} will be introduced for metadata sanitization and metadata capability handling for specific subtypes. If one of these two filters is being used for the given variables, the existing sanitize_{$object_type}_meta_{$meta_key} and auth_{$object_type}_meta_{$meta_key} will not be executed – again, this is because metadata registration for an object type and subtype overrules metadata registration for only an object type.

An updated patch with the above changes is in place to test the current approach, available on the ticket #38323.

Documentation & Recommendations

The existing register_meta() function has not yet seen wide use in REST API-related plugins, so the transition to this new behavior should be smooth. However, should developers continue to omit object subtypes or fail to properly prefix their meta keys, over time the chance of conflicts between or within plugins will increase—therefore precise documentation and education are necessary to make all of this work in the long run.

To aid the transition to the new behavior, it will be recommended to switch existing calls to register_meta() over to include an object subtype. Additionally, we will no longer encourage direct calls to register_meta() when a wrapping function like register_post_meta() is available—this is in line with how it is currently preferable to call get_post_meta() over get_metadata( 'post' ), for example. We will also re-emphasize the importance of prefixing meta keys to ensure uniqueness.

Related Issues

Something that we will want to consider in the future is how to deal with meta keys of the same name that are registered for exactly the same object type and the same subtype. Currently, the second call would simply override the data registered by the first call. Alternatively, we could make the second call fail, keeping the first one the “winner”. However, this is a general problem in many areas in core (think about register_post_type() for example), so this may not be the appropriate point to discuss such a big topic.

May 10th Meeting Agenda

Next week’s meeting on Thursday, May 10th 17:00 UTC will continue to focus on discussing register_meta(), particularly review whether the current patch is a viable solution. We will also try to find an answer to the last, smaller question of the original announcement post, whether comment types should actually be considered subtypes in scope of metadata registration. If there is time left afterwards, we can also discuss the state of the current directly Gutenberg-related work in the REST API – otherwise this will happen the week after.

Please review the register_meta() ticket #38323 and the latest patch there, and chime in with your thoughts on the ticket or by attending the meeting!

#rest-api, #summary

Completing the implementation of metadata registration with the REST API

A priority of the REST API team for the upcoming 5.0 release is to finally enhance WordPress metadata registration so that it can cover common API use-cases. This will help with adaptation of the REST API by making management of custom metadata in existing content endpoints easier, which should in turn also benefit any metadata-related work in Gutenberg. This post provides some insight into the surrounding issues, shows the current progress of the improvements, and highlights what still needs to be discussed.

A little history on register_meta()

The register_meta() function has been around since 3.3, but did not attract much attention with developers until WordPress 4.6 introduced enhancements to permit metadata registration for use within the REST API. As with other metadata functions, an object type (by core definitions either a “post”, “term”, “comment”, or “user”) needs to be passed to the function, alongside the meta key to register and the arguments for its behavior.

However, metadata in WordPress is often used only for a subset of one of the above four types. When dealing with post meta in a plugin for example, that post metadata commonly only applies to one or more specific post types – not to all posts of any post type. That is what currently makes register_meta( $object_type, $meta_key, $args ) unusable for the majority of REST API-related cases. Before the enhanced version was introduced in 4.6 many thoughts were put into this concern, with a subtype-aware implementation even being merged into core. That implementation was reverted before the release because it was unclear how to handle conflicts (more on that below). The ticket where all those changes were discussed is #35658.

Misconceptions about register_meta()

The improvements and their documentation caused developers to listen up, but unfortunately since the concept of an object type is rarely exposed in high-level WordPress APIs, many people misunderstood the first parameter as requiring a post type because post meta is the most common kind of metadata used by plugin developers. Some developers started using the function to register post type-specific meta keys without that function actually supporting that, as shown in this example:

Note:

This does not do what you may think it does:

register_meta( 'book', 'isbn', array( ... ) );

The code above will not register an ISBN number meta key for a “book” post type. It will register that key for a custom object type “book”—which probably doesn’t exist, unless a whole custom meta database table, a custom object database table, and surrounding setup has been implemented.

Those function calls fortunately don’t cause any harm in most cases since WordPress only contains four object types that use metadata (ignoring multisite). It becomes problematic though if the function is used to register metadata for the “post” post type.

Note:

This also does not do what you may think it does:

register_meta( 'post', 'some_meta_key', array( ... ) );

Since the “postpost type shares a name with the “postobject type, and since, again, the first parameter of register_meta() is for object type and not for a post type, register_meta() will register that meta key and arguments passed for all posts of any subtype (post type).

If you’re reading this and have used register_meta() in a way where you assumed it could register metadata for a single post type or taxonomy, at present we recommend you switch to using register_rest_field(), which is currently the only (however a little more involved) way to achieve this.

Adding subtype support for metadata behavior

A follow-up ticket #38323 aims to finally implement this missing functionality to register_meta() so that there’s an easy-to-use API to handle subtype-specific metadata in the REST API. It picks up the approach used before, where an additional object_subtype argument in the $args parameter can be used to specify the subtype the meta key should be registered with. Picking up the aforementioned example, you could register the ISBN for a “book” post type via the following code:

register_meta( 'post', 'isbn', array(
	'object_subtype' => 'book',
	...
) );

Open questions regarding subtype handling

Implementing meta registration for subtypes and modifying existing behavior to account for it is rather straightforward, and there is a patch available to experiment with the current approach. There are a couple things that need to be discussed though:

  • How should we deal with or prevent conflicts between meta keys of the same name and object type? The method currently allows to register metadata for an entire object type of course, so we cannot simply change that now and prevent those calls entirely. It appears we have four alternatives here:
    1. Allow meta keys of the same name and object type to be registered in any way. In case a meta key is registered for the current subtype, but also more generally for the object type, the behavior registered for the subtype takes precedence. Meta keys are internally stored nested under their object type and object subtype ($wp_meta_keys[ $object_type ][ $object_subtype ][ $meta_key ]). This is how the current patch on the ticket works, mainly because it is the most straightforward and flexible approach (it was also how the original code from #35658 worked). However it doesn’t deal with conflicts at all, so it is likely not preferable. An issue would be that on every update of a value, sanitization would happen for both behaviors, easily resulting in invalid data and unexpected errors. Throwing notices would be an option – but it would only help preventing conflicts, not actually prevent them.
    2. Allow meta keys of the same name and object type to be registered either per subtype OR for the entire object type. It would be based on whoever comes first takes priority. Meta keys are internally stored nested under their object type and object subtype ($wp_meta_keys[ $object_type ][ $object_subtype ][ $meta_key ]). If someone registers a meta key for an object type after someone else already registered the same meta key for a subtype of that object type, the process should be rejected and fail. If someone registers a meta key for an entire object type without the same meta key already being registered for a subtype of that object type, afterwards all meta keys for a subtype of that object type would be rejected. This prevents all possible conflicts since meta keys will only exist once per object type/object subtype combination, but still provides the flexibility of having multiple meta keys of the same name for different object subtypes. The implementation to prevent the conflicts during meta registration would be a little more involved though.
    3. Allow meta keys of the same name and object type to be registered only once in total. It would be based on whoever comes first takes priority. Meta keys are internally stored nested under their object type only, without an additional level for their object subtype ($wp_meta_keys[ $object_type ][ $meta_key ]). Whoever comes first, can register their meta key, every following request to add the same meta key for the same object type afterwards will be rejected. This prevents all possible conflicts since the data structure alone makes it impossible for two meta keys of the same name and object type to be registered. The approach generally appears to not be flexible enough: For example if two different post types were using a meta key of the same name, they wouldn’t be able to co-exist. However, based on the assumption that developers prefix all their metadata (which is probably not taught often enough), this option could very well work and be a straightforward solution.
    4. Allow meta keys of the same name and object type to be registered in any way, but use the behavior registered for the entire object type as a fallback only. In other words, if the same meta key exists with a subtype, only run that logic – for all subtypes where there is no specific behavior available, fall back to the behavior registered on the entire object type. To be precise, the difference between this approach and the first one is that in the first one the behavior (like sanitization and auth) would be executed for both, while here it only happens for the object subtype or the object type when no subtype-specific handling is registered. This is a conflict-free approach that at the same time doesn’t require any restrictions where calls to register_meta() would need to be rejected. The idea of using meta keys registered for an entire object type as fallback is slightly different from the other variants, but it seems to work well. It should still be highlighted that meta keys should be prefixed, but if that is a given, this implementation would even allow an individual plugin to provide more flexible logic for its metadata that may be needed for all post types (general behavior as fallback, subtype-specific behavior where needed).
  • Should we introduce easy-to-use wrapper functions like register_post_meta( $post_type, $meta_key, $args ) and register_term_meta( $taxonomy, $meta_key, $args ) which wrap register_meta() appropriately? The reason those might be useful is that the concept of object types and object subtypes is not commonly exposed in core’s documentation & high-level APIs (think about *_post_meta(), *_term_meta(), *_comment_meta() and *_user_meta() all wrapping the respective lower-level *_metadata() function). By adding these methods we would then recommend to only use those, and never recommend the use of register_meta() itself (as registering metadata for an entire object type is probably not desired). The only downside would be that comments and users don’t really have subtypes, so those two functions would either have a redundant first parameter or not have that parameter at all (which wouldn’t be future-proof).
  • A more specific issue: Should a comment’s comment_type value be considered a subtype? While one could argue that they are subtypes, they are treated entirely differently and do not really define how that comment works or how it can be edited. There’s also no flexible API around it to register additional comment types, and in the REST API there are no individual controllers per comment type. Currently, the patch ignores comment types, but this is still open for discussion.

Upcoming Meeting

The above items will be discussed in next week’s upcoming REST API meeting which will take place at the regular meeting time, Thursday, May 3rd, 2018 at 17:00 UTC in the #core-restapi channel. If you’re interested, please attend this meeting so that we can move forward with this ticket, as it should be included in WordPress 5.0. Also feel free to review the latest patch on #38323 and leave a comment on the ticket. While it is unlikely that this is the final implementation, a lot of it would not need to be adjusted regardless of the outcome for the above questions.

Extra credit goes to @kadamwhite for proofreading and several improvements.

#agenda, #rest-api, #team-update

Improvements in REST API request parameter regular expressions

With WordPress 4.9, a bug has been fixed which would cause unexpected numeric results to be included in the parsed URL parameters for a REST API request. Prior to this change, calling WP_REST_Request::get_params() for a request like /wp/v2/users/(?P<id>[\d]+) with an ID of 10 would return array( 'id' => 10, 1 => '10' ), where the latter numeric key is unnecessary and a result of PCRE matching against a named subpattern (see preg_match() documentation). The fix ensures that the above request now only returns array( 'id' => 10 ) instead. This helps for example to verify that a request does not include more than a few specific parameters.

The WP REST API docs have always been using named URL parameters, using regular (numeric) matches was never recommended. With this bug fix in place, using named parameters is now effectively required, for example /my-namespace/my-endpoint/(?P<numeric_param>[\d]+) must be used instead of /my-namespace/my-endpoint/(\d+). For background discussion on these changes, see #40704.

#4-9, #dev-notes, #rest-api

REST API Roadmap

If you’ve been following WordPress development this year, you may be wondering “what’s been happening with the REST API focus?” We’ve been a little under-the-radar for most of the year so far, so we thought publishing an update and roadmap might be a good idea.

For new contributors looking to get involved with the REST API focus or WordPress generally, there’s never been a better time, and we’d love to have your help on our projects. Read on to see what we’ve been doing, where we’re going, and how you can get involved.

Since 4.7

Since the REST API was merged into core in WordPress 4.7, development activity has unfortunately been light. The merge into core was a huge effort, and after shipping in 4.7 we saw a drop-off in contribution and overall momentum as many API contributors took a break to recover from the stress of the merge. These contributions have not returned to previous levels, and there’s a few factors behind this: the move to Trac, lack of a forward roadmap, and overall fatigue have hampered our ability to move forward quickly.

The core REST API focus goal is to utilise the REST API within the WordPress admin. Defining the scope of this project has involved auditing all admin-ajax calls, as well as the filters used inside these calls, and where they are used. In addition, we’ve been working on the low-level JavaScript utilities we need to offer conceptual compatibility: while we can deprecate and remove old PHP filters, we need to offer new JS-based filters to replace them.

The admin-ajax audit revealed that the majority of ajax requests can be grouped into four categories: Media, Themes, the Editor, and List Tables. The code for both Media and the Editor will gradually switch to the REST API moving forward with development around Gutenberg, and likewise endpoints for better managing Themes are expected to be incorporated into Customizer work.

Rather than simply refactor the existing code for the other actions in a piecemeal fashion, we’ve been working on prototyping bigger groups of related changes and features, starting with the New List Tables and Live Settings.

A significant portion of existing admin-ajax code is for handling list table actions. The existing JS code for these actions is particularly difficult to work with, and the existing list table actions user experience is frustratingly inconsistent (for example, deleting a comment happens inline, whereas deleting a post causes a page refresh). A reworking of the code has the potential to improve UX significantly. New List Tables allows us to explore ideas around how we can improve the content organisation and management experience in the admin. This is a prototyping plugin where we’ve been exploring backwards compatibility techniques, and thinking about how a theoretical new management interface would look.

The Live Settings prototype uses the REST API settings endpoints to add live saving to the Settings screens. This dovetails with the work underway by the Accessibility team to switch to the Settings API, and the two projects will be able to work together in the future.

In addition to these API team projects, work has continued on REST API-related pieces on other teams, notably the Customizer and Multisite teams, who are working on API endpoints in their respective components.

Renewing our focus

Moving forward with the REST API, there’s a few key items we’re going to be focussing on. These items will have their own dedicated subteams and development cycles, and will work in parallel. The two broad goals are to use the API in the admin, and to solve authentication for external applications.

API in the Admin

Getting the API used in the WordPress admin is our primary focus. While it is technically possible to directly switch from admin-ajax calls to REST API calls, this is essentially refactoring with no real user benefit. Instead, we want to focus on changes that can improve the user experience.

For the feature prototypes (New List Tables, and Live Settings), we’re engaging members of the Design team to lead these features from a UX perspective. So far, these prototypes have been primarily about proving out the features and ensuring it’s actually technically possible to migrate these features to use the REST API; with the initial success on the technical side, we need to switch focus to delivering compelling user experiences.

New List Tables will be working with the goal of prototyping improved content management, using the REST API. This includes unifying and standardising existing interactions, and improving the perceived performance. This is a React-based prototype, and uses the existing REST API endpoints.

Live Settings will be working with the goal of making settings changes seamless. We’ve seen huge strides forward with the Customizer for updating your site, and the backend settings deserve a better experience to match. Live Settings touches on similar areas as the Settings API Enhanced project spearheaded by the Accessibility team; we plan to continue working independently to avoid blocking each other, while keeping in touch about the respective projects.

Work on converting existing admin-ajax code to use the REST API will continue, however this won’t be a priority, as it generally doesn’t provide a strong benefit to end users. Most admin-ajax actions will naturally be deprecated as part of progress by other focuses, including Gutenberg and plans around the Media Library. We’ll continue working with other teams and focuses on their efforts here.

Authentication

External authentication is an unsolved problem, and one that’s crucial to API use outside of WordPress core itself, including the official WordPress apps.

There are two key problems to solve here: how do apps act on behalf of a user (authentication), and how do sites recognise valid apps (initial connection). We have existing solutions to both these problems (OAuth 1.0a and the Broker system respectively), however these are not the easiest solutions, and aren’t adequate for all use cases.

To date the most complete authentication solution maintained by the REST API team has been a plugin providing OAuth 1.0a authentication. Moving forward, we are switching authentication focus to OAuth 2. As Matt announced last year, we are going to begin shipping HTTPS-only features in WordPress: this allows us to switch to OAuth 2. Work started during the WordCamp Europe contributor day on a new official OAuth 2 provider plugin which is now under ongoing development.

Simplifying the initial connection is a much harder piece, and this is a long-term project. Eventually, this should be as simple as a “Connect to WordPress” button, requiring minimal effort for app developers and no effort for users. This is a complex problem to solve, and no similar software has to work on the same scale we have. In the meantime however, we’re going to investigate pre-configuring Core to recognize and permit authentication from certain default apps, including the official WordPress mobile applications. Whitelisting applications in core is a practical expedient but this solution is not sustainable in the long-term, and should be replaced with a better system as soon as feasible. Work on solving this issue will be in the mid-term, however, as we need to ensure we have solid basics first.

Help Us!

The toughest challenge facing the REST API team right now is resourcing. There are only a few people working on the API regularly, and we need help to build out our projects—which is hopefully where you come in. We need people of all skillsets to help on New List Tables, Live Settings, and OAuth 2. This includes regular contributors, JS developers, and designers. And all of this will need documentation, too: following a productive contributor day at WordCamp Europe we are making progress expanding and reorganizing the REST API developer handbook, and would gladly welcome any interested docs contributors.

Our plan is to release the first public beta of each of these projects within the next month, with regular releases from each project following that. New List Tables and Live Settings could be part of either a 4.9 or 5.0 release, while OAuth 2 will remain as a plugin until fully proven out, and would likely target a core release next year. This also requires coordination with the Mobile team, and finalising the approach to usage inside the apps.

If you’re interested in getting involved, we’d love to get your help. The API holds weekly meetings every Wednesday at 13:00 UTC (next meeting at Wednesday at 13:00 UTC), and we’re always happy to spend time helping people get started contributing.

#rest-api, #roadmap

Approaches for a complete sites endpoint and an expanded WP_Site_Query

Multisite office hours are held every Tuesday at 16:00 UTC in #core-multisite. The next will be Tuesday 16:00 UTC.

Creating a useful sites/ endpoint for the REST API and making WP_Site_Query more useful have been frequent topics over the last few weeks in #core-multisite. Quite a bit of discussion has been centered around the idea of a global wp_blogmeta table, whether it’s a good fit for core, and (if so) how to approach its introduction. See #37923 and the previous make/core post discussing a sites endpoint for additional background.

The intent of this post is to step back a bit and clarify the issues at hand to help identify the right solution(s).

The initial site information problem

A request to a global wp-json/wp/v2/sites/ endpoint on a network should return a set of site objects, similar to how wp-json/wp/v2/posts/ returns a set of post objects. A request to wp-json/wp/v2/sites/4 would return a single site object.

Written today each site object, represented as WP_Site in PHP, would provide at least this data:

  • ID
  • domain
  • path
  • registered
  • last_updated
  • public
  • archived
  • mature
  • spam
  • deleted
  • lang_id

The above fields all mirror what is available in the global wp_blogs database table installed with multisite. These are useful on their own, but additional data is frequently required.

Example: One piece of the admin that would be great to power with the REST API is the My Sites menu that multisite users see in the toolbar. To build this view, the home URL, admin URL, and site name are all necessary. In PHP, this data is made available through magic properties on the WP_Site object. When home, siteurl, blogname, or post_count is requested, the site uses switch_to_blog() and get_option() to populate the data from the individual site’s options table. If 25 sites are on the list, this generates 25 context switches and accesses 25 different tables.

There are at least a few approaches here:

  • Mirror the existing PHP experience and ensure properties are populated before the REST API response is returned. Possibly introduce a lighter weight switch_to_blog() option.
  • Provide a basic site object as well as an option to _embed other data in the response.
  • Migrate these properties into wp_blogs from wp_#_options as additional columns.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

The querying sites problem

It’s currently possible to query for sites by the default fields listed above. This data is useful for querying, but it would also be nice to query by a site’s name, description, or other piece of data at a global level.

Example: In the list table that displays all the sites of a network, it’s possible to query by a site’s domain or path, but not by it’s actual name. In a large network, a user may find it difficult to find a site when many sites share similar domains or paths. The user may know the site’s title, but not the address itself.

There is no real workaround for this issue in core right now as each site’s name is stored individually in that site’s wp_#_options table and cannot be queried collectively.

Possible approaches:

  • Migrate these properties into wp_blogs() from wp_#_options as additional fields.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

Feedback please

Please leave any thoughts you have on possible approaches to these 2 problems. It would be especially helpful to identify some use cases that may not yet be clear, or approaches that are not listed above. All feedback is welcome. 🙂

#multisite, #rest-api

Improving the REST API users endpoint for multisite in 4.7.3 and 4.8

Multisite office hours are held every Tuesday at 17:00 UTC in #core-multisite. The next will be Tuesday 17:00 UTC.

Improving the users endpoint was the main focus of this week’s office hours. The following is a recap of the decisions from that discussion. Please leave your feedback in the comments on this post. Related meeting notes are available from the January 10th office hours and the January 3rd office hours.

Chat log

Attendees: @iamfriendly, @sergeybiryukov, @flixos90, @ssstofff, @vizkr, @jnylen0, @nerrad, @johnbillion, @jeremyfelt

4.7.3

Some small changes to the users endpoint for multisite should be made in 4.7.3. The ticket for these changes is #39701.

  • Fail when a GET request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.
  • Fail when a PUT request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

The expectation for the users endpoint in 4.7.3 is that only users from the current site can be listed or managed in any way via the REST API. Global access to users will not be available, even to global administrators (super admin).

4.8

The users endpoint will be improved significantly for the 4.8 release, ideally providing full compatibility with how users are currently managed in a multisite configuration. The ticket for this task is #39544.

Here are the expectations for the users endpoint in 4.8:

  • POST to site.com/wp/v2/users/ with an existing global user’s email address adds the existing global user to a site.
  • POST to site.com/wp/v2/users/ with complete new user data creates a new global user.
  • GET to site.com/wp/v2/users/ with a global parameter set to true lists all global users.
  • GET to site.com/wp/v2/users/ without a global parameter lists only the site’s users.
  • GET to site.com/wp/v2/users/<id> with a global parameter set to true lists data for a global user, even if they are not a member of the site.
  • GET to site.com/wp/v2/users/<id> without a global parameter lists data for a user only if they are a member of the site.
  • PUT to site.com/wp/v2/users/<id> with a global parameter set to true updates a global user and data for a user that is in the global context.
  • PUT to site.com/wp/v2/users/<id> without a global parameter updates a data for a site user that is in the site context. This is how role data can be passed to maintain a user’s relationship with the site.
  • DELETE to site.com/wp/v2/users/<id> with a global parameter set to true deletes the user completely.
  • DELETE to site.com/wp/v2/users/<id> without a global parameter removes the user from the site.

Note that while users exist in a global context, data attached to these users can exist in the global context (e.g. email address) or in a site context (e.g. site role). There may need to be a method for registering user meta in a way that specifies whether it should be treated as global or site data.

Much of the work for 4.8 is still fluid. Please leave feedback on this approach in the comments and join office hours in #core-multisite on Tuesday 17:00 UTC!

#multisite, #networks-sites, #rest-api

Providing a REST API sites endpoint for multisite

One of the objectives for multisite is to determine how sites can be managed with the REST API. This has been an agenda item for the last two weeks and quite a bit has been processed. This will continue to be a topic, so please stop in for #core-multisite office hours on Tuesday 17:00 UTC and please leave your feedback in the comments on this post.

January 17 chat log and January 24 chat log in #core-multisite

Attendees: @kenshino, @vizkr, @danhgilmore, @iamfriendly, @flixos90, @schlessera, @sergeybiryukov, @pbearne, @paaljoachim, @streetlamp, @jnylen0, @loreleiaurora, @maximeculea

The requirements for the /sites/ endpoint can be summed up with these assertions:

  • The /sites/ endpoint should provide a useful set of data for each site without requiring the use of switch_to_blog().
  • It should be possible to query /sites/ for something other than ID, domain, and path.

In its current state, any /sites/ endpoint is limited to the fields in the wp_blogs table. Data such as a site’s name and description are stored in each individual site’s wp_#_options table.

Given a list of 20 sites, switch_to_blog() will be used 20 times so that get_option() can be used to retrieve things like home, siteurl, blogname, and blogdescription. An example of how inefficient this is can be found in the generation of the My Sites menu. Caching has gotten better with the introduction of WP_Site and WP_Site_Query, but there is an opportunity to change how the information is stored.

In #37923, @johnjamesjacoby suggests the introduction of a wp_blogmeta table that provides access to some site information in a common table. After discussing this in the January 17 chat, @iamfriendly and @flixos90 agreed to each take a look at core’s default options stored in wp_options and determine which made sense in a shared wp_blogmeta table. Richard’s results can be found in a comment on the ticket and Felix’s in the Slack discussion.

After some discussion in the January 24 chat, that list can be pared down a bit more.

  • home
  • siteurl
  • blogname
  • blogdescription
  • admin_email

The creation of a new table, wp_blogmeta, and migration of data for each site from wp_#_options is something that needs feedback. Without this change, a /sites/ endpoint is still possible, but may be limited. With the change, a /sites/ endpoint is much more useful, but requires a careful migration process.

#multisite, #networks-sites, #rest-api

Controlling access to REST API user functionality for multisite

Following last week’s discussion in #core-multisite (read the recap) this week’s office hours agenda was to continue the chat about the multisite-related enhancements for the REST API users endpoint, focussing heavily on how to access the required functionality. Here is a wrap-up of the discussion.

Chat log in #core-multisite

Attendees: @jeremyfelt, @jnylen, @nerrad, @ipstenu, @earnjam, @kenshino, @maximeculea, @mikelking, @lukecavanagh, @flixos90

Now that the way on how one should be able to modify user roles per site was clarified last week, this week the focus laid on where one should be able to perform those actions. The current state of the wp-json/wp/v2/users endpoint in multisite is:

  • The users overview accessible with a GET request to wp-json/wp/v2/users only lists users that are part of the current site.
  • When creating a user with a POST request to wp-json/wp/v2/users, that user is created and added to the current site. When providing the roles parameter, the passed roles are added to the user, otherwise the user will still be part of the site, but without any role. See #38526 for background.
  • It is possible to both read and edit any user from any site with a request to wp-json/wp/v2/users/<id>, regardless of whether the user is part of that site.
  • A DELETE request to wp-json/wp/v2/users/<id> results in an error. See #38962 for background.

After the discussion about how to be able to add a specific user to a site, update their site capabilities and remove them from a site, this week’s chat revolved around where these actions can be accessed, as they are for the most part network-specific actions not available to a site administrator. The approach that was agreed on is:

  • The users overview at wp-json/wp/v2/users should continue to only show users of that site by default, but a request like wp-json/wp/v2/users?global=true should show all users in the WordPress setup. This parameter must only be available to network administrators though, more specifically users with the manage_network_users capability. In the future a network parameter might also be introduced for support of multi networks, but at this point core does not support querying users per network. Accessing global users should be available from all sites in a setup instead of only from the main site. While this approach makes these endpoints duplicates of each other, it has several benefits like preventing the requirement for cross-domain requests, allowing easier API discovery and not requiring the main site of a setup to be exposed to REST API calls to a sub site.
  • Assigning an existing user to a site and removing a user from a site should generally be only available to network administrators, and the site administrators of the site that is being interacted with.
  • Similarly, editing a user that does not belong to the current site should only be possible for a network administrator. Currently this is available to site administrators as well which is probably wrong.
  • Deleting any user completely should only be available to a network administrator. A good way to handle the reassign parameter needs to be found though.

Before coming to the conclusion that dealing with multisite functionality at the existing users endpoint, the possibility of introducing a multisite-specific endpoint only available on the main site was discussed. However this was considered not practical due to the nature of how users work in WordPress. Having separate endpoints for other network-wide functionality might still be a possibility though as long as that component solely affects the network admin, so this idea is something to keep in mind for thought about further network functionality endpoints in the future.

Back to the users endpoint, one related question that came up is:

  • Should the sites a user belongs to be available at the wp-json/wp/v2/users endpoint or at a future wp-json/wp/v2/sites endpoint? If they were available in the wp-json/wp/v2/users endpoint, every user entity would have a new sites key available if the current user had sufficient permissions to see these. If they were available in the wp-json/wp/v2/sites endpoint, that endpoint could easily support this functionality through usage of a user parameter.

@jeremyfelt suggested to look at the “Add New User” screen in the site admin to have a good use-case for how to scaffold the multisite functionality of the API endpoint. This has helped during this week’s office-hours and can also be beneficial in the future. Eventually this screen should be revamped entirely, being powered by the REST API.

Regarding the enhancements of the users endpoint, a general ticket for this task was opened at #39544. This ticket is meant to be used for discussion on the topic, while separate smaller tickets should be opened for actually implementing the individual pieces. For now feedback is welcome on that ticket. The discussion on multisite improvements for the REST API will continue at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

 

 

#multisite, #networks-sites, #rest-api, #users

Improving the REST API users endpoint in multisite

One of the objectives for multisite is sorting how users are managed with the REST API. This was one of the agenda items for last week’s #core-multisite office hours and generated some good discussion. Here’s a wrap-up of the ideas and thoughts from that discussion.

Chat log in #core-multisite

Attendees: @iamfriendly, @johnjamesjacoby, @nerrad, @florian-tiar, @mikelking, @earnjam, @jeremyfelt

Users in multisite exist globally and are shared among sites on one or more networks. Users are associated with sites in the user meta table with a wp_#_capabilities key.

The current state of the wp-json/wp/v2/users endpoint for multisite is:

  • A POST request for a new global user to the main site creates the user and associates them with the main site only.
  • A POST request for a new global user to a sub site creates the user and associates them with the sub site only.
  • A POST request for an existing global user results in an error.
  • A PUT request for an existing global user to a sub site updates the user’s meta with a capability for that sub site.
  • A DELETE request on multisite is invalid and results in an error. See #38962.

It is not possible to remove a user from an individual site or to delete the user from the network.

Previous tickets: #38526, #39155, #38962, #39000

The following are a few thoughts expressed separately from the above summary.

  • The right way to associate existing objects over the REST API is with a PUT request.
  • The right way to disassociate existing objects is with a PUT request.
  • Linked previous discussion – “Deleting an item should always delete an item
  • We already have functions like remove_user_from_blog() and add_user_to_blog() available to us.
  • Does “add” invite or literally add? This can probably be included as data in the PUT request.
  • What happens if an API client is built for single site and then that site gets switched to multisite?
  • Handling bulk actions on an endpoint would be nice. (e.g. Add a user to multiple sites) No endpoint has implemented batch handling yet though.

Initial tasks:

  • It should be possible to remove a user from a site with a PUT request to the wp-json/wp/v2/users/# endpoint.
  • It should be possible to delete a global user with a DELETE request to the wp-json/wp/v2/users/# endpoint once all sites have been disassociated.

New tickets will be created soon for these tasks. Please leave any initial feedback in the comments on this post covering the assumptions and conclusions made above. There will be another round of discussion during tomorrow’s #core-multisite office hours at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

#multisite, #networks-sites, #rest-api, #users