Multisite Focused Changes in 4.8

Here’s an overview of the developer facing changes made in multisite for the 4.8 cycle. If you’re interested in more detail, checkout the full list of tickets.

New capabilities

As part of the goal to remove all calls to is_super_admin() in favor of capability checks, two new meta capabilities have been introduced in this release:

  • upgrade_network is used to determine whether a user can access the Network Upgrade page in the network admin. Related to this, the capability is also checked to determine whether to show the notice that a network upgrade is required. The capability is not mapped, so it is only granted to network administrators. See #39205 for background discussion.
  • setup_network is used to determine whether a user can setup multisite, i.e. access the Network Setup page. Before setting up a multisite, the capability is mapped to the manage_options capability, so that it is granted to administrators. Once multisite is setup, it is mapped to manage_network_options, so that it is granted to network administrators. See #39206 for background discussion.

Both capabilities work in a backward-compatible fashion, so no changes should be required in plugins that deal with those areas. The capabilities however allow more fine-grained control about access to their specific areas. As an additional reminder: Usage of is_super_admin(), while not throwing a notice, is discouraged. If you need to check whether a network administrator has access to specific functionality in your plugin, use one of the existing network capabilities or a custom capability. See #37616 for the ongoing task.

New Hooks

  • deleted_blog fires after a site has been deleted. See #25584
  • signup_site_meta filters site meta in wpmu_signup_blog(). See #39223
  • signup_user_meta filters user meta in wpmu_signup_user(). See #39223
  • minimum_site_name_length filters the minimum number of characters that can be used for new site signups. See #39676

Network-specific site and user count control

With WordPress 4.8, several functions related to network site and user counts now support an additional $network_id parameter that allows to read and update those counts on a different network than the current one. While this mostly increases general flexibility, the new parameter can be leveraged to fix inconsistencies, for example prior to 4.8 the creation of a new site on another network would falsely trigger a recalculation of the current network instead of the network being actually modified. See #38699 for the bug ticket and resulting task.

The functions that now support a $network_id parameter are:

  • get_blog_count(). See #37865.
  • get_user_count(). See #37866.
  • wp_update_network_site_counts(). See #37528.
  • wp_update_network_user_counts(). See #40349.
  • wp_update_network_counts(). See #40386.
  • wp_maybe_update_network_site_counts(). See #40384.
  • wp_maybe_update_network_user_counts(). See #40385.
  • wp_is_large_network(). The filter wp_is_large_network now passes the network ID as fourth parameter as well. See #40489.

#4-8, #dev-notes, #multisite

Approaches for a complete sites endpoint and an expanded WP_Site_Query

Multisite office hours are held every Tuesday at 16:00 UTC in #core-multisite. The next will be Tuesday 16:00 UTC.

Creating a useful sites/ endpoint for the REST API and making WP_Site_Query more useful have been frequent topics over the last few weeks in #core-multisite. Quite a bit of discussion has been centered around the idea of a global wp_blogmeta table, whether it’s a good fit for core, and (if so) how to approach its introduction. See #37923 and the previous make/core post discussing a sites endpoint for additional background.

The intent of this post is to step back a bit and clarify the issues at hand to help identify the right solution(s).

The initial site information problem

A request to a global wp-json/wp/v2/sites/ endpoint on a network should return a set of site objects, similar to how wp-json/wp/v2/posts/ returns a set of post objects. A request to wp-json/wp/v2/sites/4 would return a single site object.

Written today each site object, represented as WP_Site in PHP, would provide at least this data:

  • ID
  • domain
  • path
  • registered
  • last_updated
  • public
  • archived
  • mature
  • spam
  • deleted
  • lang_id

The above fields all mirror what is available in the global wp_blogs database table installed with multisite. These are useful on their own, but additional data is frequently required.

Example: One piece of the admin that would be great to power with the REST API is the My Sites menu that multisite users see in the toolbar. To build this view, the home URL, admin URL, and site name are all necessary. In PHP, this data is made available through magic properties on the WP_Site object. When home, siteurl, blogname, or post_count is requested, the site uses switch_to_blog() and get_option() to populate the data from the individual site’s options table. If 25 sites are on the list, this generates 25 context switches and accesses 25 different tables.

There are at least a few approaches here:

  • Mirror the existing PHP experience and ensure properties are populated before the REST API response is returned. Possibly introduce a lighter weight switch_to_blog() option.
  • Provide a basic site object as well as an option to _embed other data in the response.
  • Migrate these properties into wp_blogs from wp_#_options as additional columns.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

The querying sites problem

It’s currently possible to query for sites by the default fields listed above. This data is useful for querying, but it would also be nice to query by a site’s name, description, or other piece of data at a global level.

Example: In the list table that displays all the sites of a network, it’s possible to query by a site’s domain or path, but not by it’s actual name. In a large network, a user may find it difficult to find a site when many sites share similar domains or paths. The user may know the site’s title, but not the address itself.

There is no real workaround for this issue in core right now as each site’s name is stored individually in that site’s wp_#_options table and cannot be queried collectively.

Possible approaches:

  • Migrate these properties into wp_blogs() from wp_#_options as additional fields.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

Feedback please

Please leave any thoughts you have on possible approaches to these 2 problems. It would be especially helpful to identify some use cases that may not yet be clear, or approaches that are not listed above. All feedback is welcome. 🙂

#multisite, #rest-api

Improving the REST API users endpoint for multisite in 4.7.3 and 4.8

Multisite office hours are held every Tuesday at 17:00 UTC in #core-multisite. The next will be Tuesday 17:00 UTC.

Improving the users endpoint was the main focus of this week’s office hours. The following is a recap of the decisions from that discussion. Please leave your feedback in the comments on this post. Related meeting notes are available from the January 10th office hours and the January 3rd office hours.

Chat log

Attendees: @iamfriendly, @sergeybiryukov, @flixos90, @ssstofff, @vizkr, @jnylen0, @nerrad, @johnbillion, @jeremyfelt

4.7.3

Some small changes to the users endpoint for multisite should be made in 4.7.3. The ticket for these changes is #39701.

  • Fail when a GET request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.
  • Fail when a PUT request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

The expectation for the users endpoint in 4.7.3 is that only users from the current site can be listed or managed in any way via the REST API. Global access to users will not be available, even to global administrators (super admin).

4.8

The users endpoint will be improved significantly for the 4.8 release, ideally providing full compatibility with how users are currently managed in a multisite configuration. The ticket for this task is #39544.

Here are the expectations for the users endpoint in 4.8:

  • POST to site.com/wp/v2/users/ with an existing global user’s email address adds the existing global user to a site.
  • POST to site.com/wp/v2/users/ with complete new user data creates a new global user.
  • GET to site.com/wp/v2/users/ with a global parameter set to true lists all global users.
  • GET to site.com/wp/v2/users/ without a global parameter lists only the site’s users.
  • GET to site.com/wp/v2/users/<id> with a global parameter set to true lists data for a global user, even if they are not a member of the site.
  • GET to site.com/wp/v2/users/<id> without a global parameter lists data for a user only if they are a member of the site.
  • PUT to site.com/wp/v2/users/<id> with a global parameter set to true updates a global user and data for a user that is in the global context.
  • PUT to site.com/wp/v2/users/<id> without a global parameter updates a data for a site user that is in the site context. This is how role data can be passed to maintain a user’s relationship with the site.
  • DELETE to site.com/wp/v2/users/<id> with a global parameter set to true deletes the user completely.
  • DELETE to site.com/wp/v2/users/<id> without a global parameter removes the user from the site.

Note that while users exist in a global context, data attached to these users can exist in the global context (e.g. email address) or in a site context (e.g. site role). There may need to be a method for registering user meta in a way that specifies whether it should be treated as global or site data.

Much of the work for 4.8 is still fluid. Please leave feedback on this approach in the comments and join office hours in #core-multisite on Tuesday 17:00 UTC!

#multisite, #networks-sites, #rest-api

Providing a REST API sites endpoint for multisite

One of the objectives for multisite is to determine how sites can be managed with the REST API. This has been an agenda item for the last two weeks and quite a bit has been processed. This will continue to be a topic, so please stop in for #core-multisite office hours on Tuesday 17:00 UTC and please leave your feedback in the comments on this post.

January 17 chat log and January 24 chat log in #core-multisite

Attendees: @kenshino, @vizkr, @danhgilmore, @iamfriendly, @flixos90, @schlessera, @sergeybiryukov, @pbearne, @paaljoachim, @streetlamp, @jnylen0, @loreleiaurora, @maximeculea

The requirements for the /sites/ endpoint can be summed up with these assertions:

  • The /sites/ endpoint should provide a useful set of data for each site without requiring the use of switch_to_blog().
  • It should be possible to query /sites/ for something other than ID, domain, and path.

In its current state, any /sites/ endpoint is limited to the fields in the wp_blogs table. Data such as a site’s name and description are stored in each individual site’s wp_#_options table.

Given a list of 20 sites, switch_to_blog() will be used 20 times so that get_option() can be used to retrieve things like home, siteurl, blogname, and blogdescription. An example of how inefficient this is can be found in the generation of the My Sites menu. Caching has gotten better with the introduction of WP_Site and WP_Site_Query, but there is an opportunity to change how the information is stored.

In #37923, @johnjamesjacoby suggests the introduction of a wp_blogmeta table that provides access to some site information in a common table. After discussing this in the January 17 chat, @iamfriendly and @flixos90 agreed to each take a look at core’s default options stored in wp_options and determine which made sense in a shared wp_blogmeta table. Richard’s results can be found in a comment on the ticket and Felix’s in the Slack discussion.

After some discussion in the January 24 chat, that list can be pared down a bit more.

  • home
  • siteurl
  • blogname
  • blogdescription
  • admin_email

The creation of a new table, wp_blogmeta, and migration of data for each site from wp_#_options is something that needs feedback. Without this change, a /sites/ endpoint is still possible, but may be limited. With the change, a /sites/ endpoint is much more useful, but requires a careful migration process.

#multisite, #networks-sites, #rest-api

Improving the REST API users endpoint in multisite

One of the objectives for multisite is sorting how users are managed with the REST API. This was one of the agenda items for last week’s #core-multisite office hours and generated some good discussion. Here’s a wrap-up of the ideas and thoughts from that discussion.

Chat log in #core-multisite

Attendees: @iamfriendly, @johnjamesjacoby, @nerrad, @florian-tiar, @mikelking, @earnjam, @jeremyfelt

Users in multisite exist globally and are shared among sites on one or more networks. Users are associated with sites in the user meta table with a wp_#_capabilities key.

The current state of the wp-json/wp/v2/users endpoint for multisite is:

  • A POST request for a new global user to the main site creates the user and associates them with the main site only.
  • A POST request for a new global user to a sub site creates the user and associates them with the sub site only.
  • A POST request for an existing global user results in an error.
  • A PUT request for an existing global user to a sub site updates the user’s meta with a capability for that sub site.
  • A DELETE request on multisite is invalid and results in an error. See #38962.

It is not possible to remove a user from an individual site or to delete the user from the network.

Previous tickets: #38526, #39155, #38962, #39000

The following are a few thoughts expressed separately from the above summary.

  • The right way to associate existing objects over the REST API is with a PUT request.
  • The right way to disassociate existing objects is with a PUT request.
  • Linked previous discussion – “Deleting an item should always delete an item
  • We already have functions like remove_user_from_blog() and add_user_to_blog() available to us.
  • Does “add” invite or literally add? This can probably be included as data in the PUT request.
  • What happens if an API client is built for single site and then that site gets switched to multisite?
  • Handling bulk actions on an endpoint would be nice. (e.g. Add a user to multiple sites) No endpoint has implemented batch handling yet though.

Initial tasks:

  • It should be possible to remove a user from a site with a PUT request to the wp-json/wp/v2/users/# endpoint.
  • It should be possible to delete a global user with a DELETE request to the wp-json/wp/v2/users/# endpoint once all sites have been disassociated.

New tickets will be created soon for these tasks. Please leave any initial feedback in the comments on this post covering the assumptions and conclusions made above. There will be another round of discussion during tomorrow’s #core-multisite office hours at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

#multisite, #networks-sites, #rest-api, #users

Multisite Focused Changes in 4.7

Howdy. The 4.7 release cycle has been a chance to build on some of the work from the last couple releases in multisite.  If you’d like more detail, check out the full list of multisite focused changes in this release.

get_blog_details() replaced with get_site()

A lot of progress has been made over the last few releases to get things in place for this transition. Now that WP_Site and WP_Network objects exist and are accessible with functions like get_site() and get_network(), they can be implemented throughout core.

In WordPress 4.7, get_blog_details() was replaced throughout core code with the modern  get_site(). The roadmap for this includes deprecating get_blog_details() in WordPress 4.8, so take this cycle as a chance to move your code in that direction.

get_site() is often a direct replacement, though get_sites() can also be used to query for sites when an ID is not available.

See #37102 for details on this change.

blog_details filter deprecated

In combination with the decision to stop using get_blog_details() throughout core, the (not widely used) blog_details filter has been deprecated. It has been added to get_site() to provide backward compatibility with the above change and will fire with a deprecation notice. Plugin code should use the site_details filter instead. See #38491 for details on this change.

_network_option actions and filters get $network_id

The $network_id associated with the use of a _network_option() function is now passed to the filters and actions that fire within. This provides granular control that was not available when first introduced. See #38319, #38320, #38321, and #38322 for details on this change.

wp_get_network() deprecated

It is now recommended that get_network() is used instead. See #37553 for this change.

 

 

#4-7, #dev-notes, #multisite, #network-sites

4.6.1 Release Candidate

A Release Candidate for WordPress 4.6.1 is now available. This maintenance release fixes 16 issues reported against 4.6 and is scheduled for final release on Wednesday, September 7.

Thus far WordPress 4.6 has been downloaded almost 7 million times since its release on August 16. Please help us by testing this release candidate to ensure 4.6.1 fixes the reported issues and doesn’t introduce any new ones.

All Changes

Here’s a list of all closed tickets, sorted by component:

Bootstrap/Load

  • #37680 – PHP Warning: ini_get_all() has been disabled for security reasons

Comments

  • #37696WP_Comment_Query loses sql_clauses with object cache

Database

  • #37683$collate and $charset can be undefined in wpdb::init_charset()
  • #37689 – Issues with utf8mb4 collation and the 4.6 update

Editor

  • #37690 – Backspace causes jumping

Email

  • #37736 – Emails fail on certain server setups

External Libraries

  • #37700 – Warning: curl_exec() has been disabled for security reasons (Requests library)
  • #37720 – The minified version of the Masonry shim was not updated in #37666 (Masonry library)

HTTP API

  • #37733cURL error 3: malformed for remote requests
  • #37768 – HTTP API no longer accepts integer and float values for the cookies argument

Post Thumbnails

  • #37697 – Strange behavior with thumbnails on preview in 4.6

Script Loader

  • #37800 – Close “link rel” dns-prefetch tag

Taxonomy

  • #37721 – Improve error handling of is_object_in_term in taxonomy.php

Themes

  • #37755 – Visual Editor: Weird unicode (Vietnamese) characters display on WordPress 4.6

TinyMCE

Upgrade/Install

  • #37731 – Infinite loop in _wp_json_sanity_check() during plugin install

 

#4-6-1

Bug Scrub for 4.6.1

The first bug scrub for 4.6.1 will be Tuesday 23 August, 15:00 UTC in #core.

There are currently 9 11 tickets open on the 4.6.1 milestone in various stages of verification, patching, and testing. Working with these over the next several days before the bug scrub will help determine when a 4.6.1 release will be necessary.

Any tickets reported against trunk at this point in the cycle should also be checked for issues that may have been introduced in 4.6.

All testing is helpful, so please take a look if you have time.

See you on Tuesday!

 

#4-6-1, #bug-scrub

Additional register_meta() changes in 4.6

In the last 2 weeks, the direction of register_meta() has changed significantly from the original write-up. There was a meeting to discuss some of the changes and a recap of that discussion. Some other discussion after that meeting led to a much more simplified version of register_meta() that is now shipping in 4.6.

Here’s what registering meta looked like in 4.5. This meta key has sanitization and authorization callbacks.

register_meta( 'post', 'my_meta_key', 'sanitize_my_meta_key', 'authorize_my_meta_key' );

The above code will continue to work in 4.6, though will not be considered completely registered. The callbacks will be registered, but the key will not be added to the global registry and register_meta() will return false.

Here’s what registering meta looks like in 4.6. This meta key will have sanitization and authorization callbacks, and be registered as public for the WordPress REST API.

$args = array(
    'sanitize_callback' => 'sanitize_my_meta_key',
    'auth_callback' => 'authorize_my_meta_key',
    'type' => 'string',
    'description' => 'My registered meta key',
    'single' => true,
    'show_in_rest' => true,
);
register_meta( 'post', 'my_meta_key', $args );

The above will register the meta key properly and return true.

There will no longer be a check for unique object types and subtypes for meta keys. There is no CURIE like syntax involved. Instead, be sure to uniquely prefix meta keys so that they do not conflict with others that may be registered with different arguments.

Additional helper functions get_registered_metadata(), get_registered_meta_keys(), unregister_meta(), and registered_meta_key_exists() have been added to make the innards of the global data more accessible.

The $wp_meta_keys variable should not be altered directly. It is possible that its structure will change in the future.

Any code currently using register_meta() and expecting pre-4.6 behavior will continue to work as is. Please report any breaks in compatibility that might be found.

For the full history, see #35658. 🙂

 

 

#4-6, #dev-notes, #options-meta

register_meta() Discussion Recap (July 14, 2016)

As announced yesterday, there was a discussion today covering the future of register_meta(), something that has been in progress for WordPress 4.6 in #35658. This is a recap. 🙂

Attendees: @helen, @ocean90, @rachelbaker, @mikeschinkel, @jsternberg, @sc0ttclark, @richardtape, @swissspidy, @joemcgill, @seancjones, @achbed, @jeremyfelt

Chat log.

Overview

In #35658, register_meta() uses object subtypes when registering meta so that key registration can be considered unique.

In 4.6 trunk, this information is passed as part of the 3rd argument—array( 'object_subtype' => 'books' )—to register_meta() and $object_subtype = '' has been added as an extra argument to several other pieces as a way to support that.

After exploring a bit more, it’s clear that $object_subtype will continue to spread as an additional argument throughout many _meta() functions so that registered meta is handled properly.

An alternative is to use CURIEs to describe complex meta types in a single string—'comment:reaction' rather than 'comment' and 'reaction'—so that the extra argument is not needed. Instead, processing exists to turn that string into an object type and subtype.

Examples

  • post:post
  • post:books
  • term:category
  • user:user
  • comment:reaction

The main question for today’s discussion

Is CURIE like notation the right way forward for handling complex meta types?

Decisions

After a very good and thorough conversation about the above and other points, these are the decisions for moving forward with work on #35658:

  1. Meta keys will be registered using CURIE like syntax.
    • register_meta( 'post:book', 'isbn', $args );
  2. The : used in the string to register meta keys will also be used in filters.
    • sanitize_post:book_meta_isbn
  3. Core object types will fallback to default subtypes if one is not specified.
    • post becomes post:post, comment becomes comment:comment, user becomes user:user, and term becomes term:category.
  4. Meta key registration for all/any subtypes of an object type will not be included in 4.6, but is likely something to add in the future.

Please check out the latest patches on #35658 and contribute code and thoughts on that ticket or share any questions/concerns in the discussion below.

Thanks everyone for attending today!

#4-6, #options-meta