Improving the REST API users endpoint for multisite in 4.7.3 and 4.8

Multisite office hours are held every Tuesday at 17:00 UTC in #core-multisite. The next will be Tuesday 17:00 UTC.

Improving the users endpoint was the main focus of this week’s office hours. The following is a recap of the decisions from that discussion. Please leave your feedback in the comments on this post. Related meeting notes are available from the January 10th office hours and the January 3rd office hours.

Chat log

Attendees: @iamfriendly, @sergeybiryukov, @flixos90, @ssstofff, @vizkr, @jnylen0, @nerrad, @johnbillion, @jeremyfelt

4.7.3

Some small changes to the users endpoint for multisite should be made in 4.7.3. The ticket for these changes is #39701.

  • Fail when a GET request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.
  • Fail when a PUT request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

The expectation for the users endpoint in 4.7.3 is that only users from the current site can be listed or managed in any way via the REST API. Global access to users will not be available, even to global administrators (super admin).

4.8

The users endpoint will be improved significantly for the 4.8 release, ideally providing full compatibility with how users are currently managed in a multisite configuration. The ticket for this task is #39544.

Here are the expectations for the users endpoint in 4.8:

  • POST to site.com/wp/v2/users/ with an existing global user’s email address adds the existing global user to a site.
  • POST to site.com/wp/v2/users/ with complete new user data creates a new global user.
  • GET to site.com/wp/v2/users/ with a global parameter set to true lists all global users.
  • GET to site.com/wp/v2/users/ without a global parameter lists only the site’s users.
  • GET to site.com/wp/v2/users/<id> with a global parameter set to true lists data for a global user, even if they are not a member of the site.
  • GET to site.com/wp/v2/users/<id> without a global parameter lists data for a user only if they are a member of the site.
  • PUT to site.com/wp/v2/users/<id> with a global parameter set to true updates a global user and data for a user that is in the global context.
  • PUT to site.com/wp/v2/users/<id> without a global parameter updates a data for a site user that is in the site context. This is how role data can be passed to maintain a user’s relationship with the site.
  • DELETE to site.com/wp/v2/users/<id> with a global parameter set to true deletes the user completely.
  • DELETE to site.com/wp/v2/users/<id> without a global parameter removes the user from the site.

Note that while users exist in a global context, data attached to these users can exist in the global context (e.g. email address) or in a site context (e.g. site role). There may need to be a method for registering user meta in a way that specifies whether it should be treated as global or site data.

Much of the work for 4.8 is still fluid. Please leave feedback on this approach in the comments and join office hours in #core-multisite on Tuesday 17:00 UTC!

#multisite, #networks-sites, #rest-api

Providing a REST API sites endpoint for multisite

One of the objectives for multisite is to determine how sites can be managed with the REST API. This has been an agenda item for the last two weeks and quite a bit has been processed. This will continue to be a topic, so please stop in for #core-multisite office hours on Tuesday 17:00 UTC and please leave your feedback in the comments on this post.

January 17 chat log and January 24 chat log in #core-multisite

Attendees: @kenshino, @vizkr, @danhgilmore, @iamfriendly, @flixos90, @schlessera, @sergeybiryukov, @pbearne, @paaljoachim, @streetlamp, @jnylen0, @loreleiaurora, @maximeculea

The requirements for the /sites/ endpoint can be summed up with these assertions:

  • The /sites/ endpoint should provide a useful set of data for each site without requiring the use of switch_to_blog().
  • It should be possible to query /sites/ for something other than ID, domain, and path.

In its current state, any /sites/ endpoint is limited to the fields in the wp_blogs table. Data such as a site’s name and description are stored in each individual site’s wp_#_options table.

Given a list of 20 sites, switch_to_blog() will be used 20 times so that get_option() can be used to retrieve things like home, siteurl, blogname, and blogdescription. An example of how inefficient this is can be found in the generation of the My Sites menu. Caching has gotten better with the introduction of WP_Site and WP_Site_Query, but there is an opportunity to change how the information is stored.

In #37923, @johnjamesjacoby suggests the introduction of a wp_blogmeta table that provides access to some site information in a common table. After discussing this in the January 17 chat, @iamfriendly and @flixos90 agreed to each take a look at core’s default options stored in wp_options and determine which made sense in a shared wp_blogmeta table. Richard’s results can be found in a comment on the ticket and Felix’s in the Slack discussion.

After some discussion in the January 24 chat, that list can be pared down a bit more.

  • home
  • siteurl
  • blogname
  • blogdescription
  • admin_email

The creation of a new table, wp_blogmeta, and migration of data for each site from wp_#_options is something that needs feedback. Without this change, a /sites/ endpoint is still possible, but may be limited. With the change, a /sites/ endpoint is much more useful, but requires a careful migration process.

#multisite, #networks-sites, #rest-api

Improving the REST API users endpoint in multisite

One of the objectives for multisite is sorting how users are managed with the REST API. This was one of the agenda items for last week’s #core-multisite office hours and generated some good discussion. Here’s a wrap-up of the ideas and thoughts from that discussion.

Chat log in #core-multisite

Attendees: @iamfriendly, @johnjamesjacoby, @nerrad, @florian-tiar, @mikelking, @earnjam, @jeremyfelt

Users in multisite exist globally and are shared among sites on one or more networks. Users are associated with sites in the user meta table with a wp_#_capabilities key.

The current state of the wp-json/wp/v2/users endpoint for multisite is:

  • A POST request for a new global user to the main site creates the user and associates them with the main site only.
  • A POST request for a new global user to a sub site creates the user and associates them with the sub site only.
  • A POST request for an existing global user results in an error.
  • A PUT request for an existing global user to a sub site updates the user’s meta with a capability for that sub site.
  • A DELETE request on multisite is invalid and results in an error. See #38962.

It is not possible to remove a user from an individual site or to delete the user from the network.

Previous tickets: #38526, #39155, #38962, #39000

The following are a few thoughts expressed separately from the above summary.

  • The right way to associate existing objects over the REST API is with a PUT request.
  • The right way to disassociate existing objects is with a PUT request.
  • Linked previous discussion – “Deleting an item should always delete an item
  • We already have functions like remove_user_from_blog() and add_user_to_blog() available to us.
  • Does “add” invite or literally add? This can probably be included as data in the PUT request.
  • What happens if an API client is built for single site and then that site gets switched to multisite?
  • Handling bulk actions on an endpoint would be nice. (e.g. Add a user to multiple sites) No endpoint has implemented batch handling yet though.

Initial tasks:

  • It should be possible to remove a user from a site with a PUT request to the wp-json/wp/v2/users/# endpoint.
  • It should be possible to delete a global user with a DELETE request to the wp-json/wp/v2/users/# endpoint once all sites have been disassociated.

New tickets will be created soon for these tasks. Please leave any initial feedback in the comments on this post covering the assumptions and conclusions made above. There will be another round of discussion during tomorrow’s #core-multisite office hours at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

#multisite, #networks-sites, #rest-api, #users

Multisite Focused Changes in 4.7

Howdy. The 4.7 release cycle has been a chance to build on some of the work from the last couple releases in multisite.  If you’d like more detail, check out the full list of multisite focused changes in this release.

get_blog_details() replaced with get_site()

A lot of progress has been made over the last few releases to get things in place for this transition. Now that WP_Site and WP_Network objects exist and are accessible with functions like get_site() and get_network(), they can be implemented throughout core.

In WordPress 4.7, get_blog_details() was replaced throughout core code with the modern  get_site(). The roadmap for this includes deprecating get_blog_details() in WordPress 4.8, so take this cycle as a chance to move your code in that direction.

get_site() is often a direct replacement, though get_sites() can also be used to query for sites when an ID is not available.

See #37102 for details on this change.

blog_details filter deprecated

In combination with the decision to stop using get_blog_details() throughout core, the (not widely used) blog_details filter has been deprecated. It has been added to get_site() to provide backward compatibility with the above change and will fire with a deprecation notice. Plugin code should use the site_details filter instead. See #38491 for details on this change.

_network_option actions and filters get $network_id

The $network_id associated with the use of a _network_option() function is now passed to the filters and actions that fire within. This provides granular control that was not available when first introduced. See #38319, #38320, #38321, and #38322 for details on this change.

wp_get_network() deprecated

It is now recommended that get_network() is used instead. See #37553 for this change.

 

 

#4-7, #dev-notes, #multisite, #network-sites

4.6.1 Release Candidate

A Release Candidate for WordPress 4.6.1 is now available. This maintenance release fixes 16 issues reported against 4.6 and is scheduled for final release on Wednesday, September 7.

Thus far WordPress 4.6 has been downloaded almost 7 million times since its release on August 16. Please help us by testing this release candidate to ensure 4.6.1 fixes the reported issues and doesn’t introduce any new ones.

All Changes

Here’s a list of all closed tickets, sorted by component:

Bootstrap/Load

  • #37680 – PHP Warning: ini_get_all() has been disabled for security reasons

Comments

  • #37696WP_Comment_Query loses sql_clauses with object cache

Database

  • #37683$collate and $charset can be undefined in wpdb::init_charset()
  • #37689 – Issues with utf8mb4 collation and the 4.6 update

Editor

  • #37690 – Backspace causes jumping

Email

  • #37736 – Emails fail on certain server setups

External Libraries

  • #37700 – Warning: curl_exec() has been disabled for security reasons (Requests library)
  • #37720 – The minified version of the Masonry shim was not updated in #37666 (Masonry library)

HTTP API

  • #37733cURL error 3: malformed for remote requests
  • #37768 – HTTP API no longer accepts integer and float values for the cookies argument

Post Thumbnails

  • #37697 – Strange behavior with thumbnails on preview in 4.6

Script Loader

  • #37800 – Close “link rel” dns-prefetch tag

Taxonomy

  • #37721 – Improve error handling of is_object_in_term in taxonomy.php

Themes

  • #37755 – Visual Editor: Weird unicode (Vietnamese) characters display on WordPress 4.6

TinyMCE

Upgrade/Install

  • #37731 – Infinite loop in _wp_json_sanity_check() during plugin install

 

#4-6-1

Bug Scrub for 4.6.1

The first bug scrub for 4.6.1 will be Tuesday 23 August, 15:00 UTC in #core.

There are currently 9 11 tickets open on the 4.6.1 milestone in various stages of verification, patching, and testing. Working with these over the next several days before the bug scrub will help determine when a 4.6.1 release will be necessary.

Any tickets reported against trunk at this point in the cycle should also be checked for issues that may have been introduced in 4.6.

All testing is helpful, so please take a look if you have time.

See you on Tuesday!

 

#4-6-1, #bug-scrub

Additional register_meta() changes in 4.6

In the last 2 weeks, the direction of register_meta() has changed significantly from the original write-up. There was a meeting to discuss some of the changes and a recap of that discussion. Some other discussion after that meeting led to a much more simplified version of register_meta() that is now shipping in 4.6.

Here’s what registering meta looked like in 4.5. This meta key has sanitization and authorization callbacks.

register_meta( 'post', 'my_meta_key', 'sanitize_my_meta_key', 'authorize_my_meta_key' );

The above code will continue to work in 4.6, though will not be considered completely registered. The callbacks will be registered, but the key will not be added to the global registry and register_meta() will return false.

Here’s what registering meta looks like in 4.6. This meta key will have sanitization and authorization callbacks, and be registered as public for the WordPress REST API.

$args = array(
    'sanitize_callback' => 'sanitize_my_meta_key',
    'auth_callback' => 'authorize_my_meta_key',
    'type' => 'string',
    'description' => 'My registered meta key',
    'single' => true,
    'show_in_rest' => true,
);
register_meta( 'post', 'my_meta_key', $args );

The above will register the meta key properly and return true.

There will no longer be a check for unique object types and subtypes for meta keys. There is no CURIE like syntax involved. Instead, be sure to uniquely prefix meta keys so that they do not conflict with others that may be registered with different arguments.

Additional helper functions get_registered_metadata(), get_registered_meta_keys(), unregister_meta(), and registered_meta_key_exists() have been added to make the innards of the global data more accessible.

The $wp_meta_keys variable should not be altered directly. It is possible that its structure will change in the future.

Any code currently using register_meta() and expecting pre-4.6 behavior will continue to work as is. Please report any breaks in compatibility that might be found.

For the full history, see #35658. 🙂

 

 

#4-6, #dev-notes, #options-meta

register_meta() Discussion Recap (July 14, 2016)

As announced yesterday, there was a discussion today covering the future of register_meta(), something that has been in progress for WordPress 4.6 in #35658. This is a recap. 🙂

Attendees: @helen, @ocean90, @rachelbaker, @mikeschinkel, @jsternberg, @sc0ttclark, @richardtape, @swissspidy, @joemcgill, @seancjones, @achbed, @jeremyfelt

Chat log.

Overview

In #35658, register_meta() uses object subtypes when registering meta so that key registration can be considered unique.

In 4.6 trunk, this information is passed as part of the 3rd argument—array( 'object_subtype' => 'books' )—to register_meta() and $object_subtype = '' has been added as an extra argument to several other pieces as a way to support that.

After exploring a bit more, it’s clear that $object_subtype will continue to spread as an additional argument throughout many _meta() functions so that registered meta is handled properly.

An alternative is to use CURIEs to describe complex meta types in a single string—'comment:reaction' rather than 'comment' and 'reaction'—so that the extra argument is not needed. Instead, processing exists to turn that string into an object type and subtype.

Examples

  • post:post
  • post:books
  • term:category
  • user:user
  • comment:reaction

The main question for today’s discussion

Is CURIE like notation the right way forward for handling complex meta types?

Decisions

After a very good and thorough conversation about the above and other points, these are the decisions for moving forward with work on #35658:

  1. Meta keys will be registered using CURIE like syntax.
    • register_meta( 'post:book', 'isbn', $args );
  2. The : used in the string to register meta keys will also be used in filters.
    • sanitize_post:book_meta_isbn
  3. Core object types will fallback to default subtypes if one is not specified.
    • post becomes post:post, comment becomes comment:comment, user becomes user:user, and term becomes term:category.
  4. Meta key registration for all/any subtypes of an object type will not be included in 4.6, but is likely something to add in the future.

Please check out the latest patches on #35658 and contribute code and thoughts on that ticket or share any questions/concerns in the discussion below.

Thanks everyone for attending today!

#4-6, #options-meta

register_meta() discussion on Thursday, July 14

Howdy!

There will be a meeting in #core at Thursday July 14, 19:00 UTC to discuss the changes in register_meta() for the 4.6 release. The dev notes have been published, but there is still work to be done to ensure things are great.

Of particular interest is a proposal to change to a CURIE like notation when working with the registration of meta. Instead of using two arguments (object type and subtype), there would be one argument containing both.

As an example, If a post type was registered with books as the slug and ISBN was stored as sanitized meta:

// In WordPress 4.5
register_meta( 'post', 'isbn', 'my_sanitize_callback' );

// In current trunk for 4.6
register_meta( 'post', 'isbn', array( 'object_subtype' => 'books', 'sanitize_callback' => 'my_sanitize_callback' ) );

// After proposed change
register_meta( 'post:books', 'isbn', array( 'sanitize_callback' => 'my_sanitize_callback' ) );

Please stop in #core at Thursday July 14, 19:00 UTC tomorrow if you are interested. If you can’t make it, please leave a comment on this post with your thoughts.

#4-6, #fields-api, #options-meta

Multisite Focused Changes in 4.6

The 4.6 release cycle has been a productive one! Several major and a few minor updates are described in detail below. If you’d like even more detail, here is a full list of multisite focused changes for 4.6.

Introduce WP_Site_Query and WP_Network_Query

WordPress 4.6 introduces WP_Site_Query and WP_Network_Query as well as their companion functions, get_sites() and get_networks().

WP_Site_Query

With WP_Site_Query (or get_sites()), sites can now be queried from the $wpdb->blogs table in a flexible way by id, domain, path, and more. For a full list of supported arguments, see the documentation attached to the __construct() method.

New filters and actions associated with this change include parse_site_query, pre_get_sites, the_sites, site_search_columns, sites_clauses, and found_sites_query.

Query results are cached as part of the global sites group. This will provide a performance boost for sites using a persistent object cache.

The introduction of this class helped resolve several long standing issues with WP_MS_Sites_List_Table. Expect to see a faster and more relevant search when browsing through a network’s sites. See #36675.

For the background discussion around these changes, see #35791.

WP_Network_Query

With WP_Network_Query (or get_networks()), networks can now be queried from the $wpdb->site table by id, domain, and path. Query results are cached as part of the global networks group. For a full list of supported arguments, see the documentation attached to the __construct() method.

WordPress core does not provide a UI for managing multiple networks, but any sites using a plugin to do this will start to see a benefit from having this available.

New filters and actions associated with this change include parse_network_query, pre_get_networks, the_networks, networks_clauses, and found_networks_query.

For the background discussion around these changes, see #32504.

Enhancements to WP_Site and WP_Network

New utility methods

New functions get_site() and get_network() have been introduced to retrieve a specific site or network respectively. They both accept either an ID, database object or class instance as the first parameter. If that parameter is omitted, the current site / network is returned.

Property changes on WP_Site and WP_Network

The WP_Site and WP_Network objects now support access to their respective ID properties through property names which match current naming conventions. Furthermore, by accessing the properties with the new names, it is ensured that these are returned as integers rather than strings. The old properties are still accessible and valid, but these changes should encourage developers to use the new property names.

The following is the list of property improvements:

  • $site->id is the site’s ID. (previously $site->blog_id)
  • $site->network_id is the site’s parent network ID. (previously $site->site_id)
  • $network->id is now an integer instead of a string.
  • $network->site_id is the network’s main site ID. (previously $network->blog_id)

For background discussion around these changes, see #36717 and #37050.

Lazy-loading extended properties

Additional properties of a site object normally accessed with get_blog_details() are now automatically lazy-loaded when requested. This allows developers to use get_site() as a replacement for get_blog_details() which is likely to be deprecated in a future release. See #36935.

New Actions and Filters

Several new actions and filters were added as part of the WP_Site_Query and WP_Network_Query changes. Here are some others of note:

  • ms_loaded fires at the end of the multisite bootstrap process in ms-settings.php. At this point the current site and network are available in the global scope. See #37235.
  • pre_get_blogs_of_user allows developers to alter or short-circuit the results of get_blogs_of_user(), which can be an expensive function to run on configurations with many users. See #36707.
  • clean_site_cache fires after cache has been cleared with clear_blog_cache(). This can be useful for clearing any custom cache keys associated with a site record. See #36203.
  • network_edit_site_nav_links filters the list of links displayed at the top of Edit Site views as a “tabbed” interface. See #15800.
  • ms_sites_list_table_query_args filters the arguments passed to WP_Site_Query when building the MS Sites List Table. See #26580.

Other enhancements of note

Introduce get_current_network_id() as a helper function to find the current network’s ID. See #33900.

The MULTISITE and SUBDOMAIN_INSTALL constants can now be overridden in a project’s wp-config.php when running unit tests. See #36567.

Deprecated Functions

wp_get_sites() has been deprecated and get_sites() should be used instead. Please note when making code changes that get_sites() returns an array of WP_Site objects where wp_get_sites() returned an array of arrays.

WP Multi Network compatibility

The introduction of get_networks() in 4.6 conflicts with the function of the same name in WP Multi Network, a plugin commonly used to provide multiple networks on a multisite installation. A fix for this is already in the master branch of the plugin, and will be included in the next release. In the same change, many other functions were wrapped in function_exists() calls to prevent similar conflicts in the future. If you are using WP Multi Network, please be sure to update accordingly.

#4-6, #dev-notes, #multisite, #network-sites