Multisite Focused Changes in 4.8

Here’s an overview of the developer facing changes made in multisite for the 4.8 cycle. If you’re interested in more detail, checkout the full list of tickets.

New capabilities

As part of the goal to remove all calls to is_super_admin() in favor of capability checks, two new meta capabilities have been introduced in this release:

  • upgrade_network is used to determine whether a user can access the Network Upgrade page in the network admin. Related to this, the capability is also checked to determine whether to show the notice that a network upgrade is required. The capability is not mapped, so it is only granted to network administrators. See #39205 for background discussion.
  • setup_network is used to determine whether a user can setup multisite, i.e. access the Network Setup page. Before setting up a multisite, the capability is mapped to the manage_options capability, so that it is granted to administrators. Once multisite is setup, it is mapped to manage_network_options, so that it is granted to network administrators. See #39206 for background discussion.

Both capabilities work in a backward-compatible fashion, so no changes should be required in plugins that deal with those areas. The capabilities however allow more fine-grained control about access to their specific areas. As an additional reminder: Usage of is_super_admin(), while not throwing a notice, is discouraged. If you need to check whether a network administrator has access to specific functionality in your plugin, use one of the existing network capabilities or a custom capability. See #37616 for the ongoing task.

New Hooks

  • deleted_blog fires after a site has been deleted. See #25584
  • signup_site_meta filters site meta in wpmu_signup_blog(). See #39223
  • signup_user_meta filters user meta in wpmu_signup_user(). See #39223
  • minimum_site_name_length filters the minimum number of characters that can be used for new site signups. See #39676

Network-specific site and user count control

With WordPress 4.8, several functions related to network site and user counts now support an additional $network_id parameter that allows to read and update those counts on a different network than the current one. While this mostly increases general flexibility, the new parameter can be leveraged to fix inconsistencies, for example prior to 4.8 the creation of a new site on another network would falsely trigger a recalculation of the current network instead of the network being actually modified. See #38699 for the bug ticket and resulting task.

The functions that now support a $network_id parameter are:

  • get_blog_count(). See #37865.
  • get_user_count(). See #37866.
  • wp_update_network_site_counts(). See #37528.
  • wp_update_network_user_counts(). See #40349.
  • wp_update_network_counts(). See #40386.
  • wp_maybe_update_network_site_counts(). See #40384.
  • wp_maybe_update_network_user_counts(). See #40385.
  • wp_is_large_network(). The filter wp_is_large_network now passes the network ID as fourth parameter as well. See #40489.

#4-8, #dev-notes, #multisite

Approaches for a complete sites endpoint and an expanded WP_Site_Query

Multisite office hours are held every Tuesday at 16:00 UTC in #core-multisite. The next will be Tuesday 16:00 UTC.

Creating a useful sites/ endpoint for the REST API and making WP_Site_Query more useful have been frequent topics over the last few weeks in #core-multisite. Quite a bit of discussion has been centered around the idea of a global wp_blogmeta table, whether it’s a good fit for core, and (if so) how to approach its introduction. See #37923 and the previous make/core post discussing a sites endpoint for additional background.

The intent of this post is to step back a bit and clarify the issues at hand to help identify the right solution(s).

The initial site information problem

A request to a global wp-json/wp/v2/sites/ endpoint on a network should return a set of site objects, similar to how wp-json/wp/v2/posts/ returns a set of post objects. A request to wp-json/wp/v2/sites/4 would return a single site object.

Written today each site object, represented as WP_Site in PHP, would provide at least this data:

  • ID
  • domain
  • path
  • registered
  • last_updated
  • public
  • archived
  • mature
  • spam
  • deleted
  • lang_id

The above fields all mirror what is available in the global wp_blogs database table installed with multisite. These are useful on their own, but additional data is frequently required.

Example: One piece of the admin that would be great to power with the REST API is the My Sites menu that multisite users see in the toolbar. To build this view, the home URL, admin URL, and site name are all necessary. In PHP, this data is made available through magic properties on the WP_Site object. When home, siteurl, blogname, or post_count is requested, the site uses switch_to_blog() and get_option() to populate the data from the individual site’s options table. If 25 sites are on the list, this generates 25 context switches and accesses 25 different tables.

There are at least a few approaches here:

  • Mirror the existing PHP experience and ensure properties are populated before the REST API response is returned. Possibly introduce a lighter weight switch_to_blog() option.
  • Provide a basic site object as well as an option to _embed other data in the response.
  • Migrate these properties into wp_blogs from wp_#_options as additional columns.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

The querying sites problem

It’s currently possible to query for sites by the default fields listed above. This data is useful for querying, but it would also be nice to query by a site’s name, description, or other piece of data at a global level.

Example: In the list table that displays all the sites of a network, it’s possible to query by a site’s domain or path, but not by it’s actual name. In a large network, a user may find it difficult to find a site when many sites share similar domains or paths. The user may know the site’s title, but not the address itself.

There is no real workaround for this issue in core right now as each site’s name is stored individually in that site’s wp_#_options table and cannot be queried collectively.

Possible approaches:

  • Migrate these properties into wp_blogs() from wp_#_options as additional fields.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

Feedback please

Please leave any thoughts you have on possible approaches to these 2 problems. It would be especially helpful to identify some use cases that may not yet be clear, or approaches that are not listed above. All feedback is welcome. 🙂

#multisite, #rest-api

Weekly Multisite Bug Scrubs

The multisite team has recently been focussed on figuring out and planning the implementation of REST API support for multisite features, which has been the primary topic during the weekly office hours. Unfortunately discussion about regular bugs and small enhancements has been a bit on hold because of that.

Therefore we would like to start running weekly bug scrubs again, and in Tuesday’s meeting we decided to have these every Monday at 18:00 UTC. We will continue to primarily focus on the REST API during office hours, so there will be a clear separation of concerns.

Please join us in #core-multisite if you’re interested or have a bug to discuss. The initial bug scrub will be on Monday 18:00 UTC.

#bug-scrub, #multisite, #networks-sites

Improving the REST API users endpoint for multisite in 4.7.3 and 4.8

Multisite office hours are held every Tuesday at 17:00 UTC in #core-multisite. The next will be Tuesday 17:00 UTC.

Improving the users endpoint was the main focus of this week’s office hours. The following is a recap of the decisions from that discussion. Please leave your feedback in the comments on this post. Related meeting notes are available from the January 10th office hours and the January 3rd office hours.

Chat log

Attendees: @iamfriendly, @sergeybiryukov, @flixos90, @ssstofff, @vizkr, @jnylen0, @nerrad, @johnbillion, @jeremyfelt

4.7.3

Some small changes to the users endpoint for multisite should be made in 4.7.3. The ticket for these changes is #39701.

  • Fail when a GET request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.
  • Fail when a PUT request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

The expectation for the users endpoint in 4.7.3 is that only users from the current site can be listed or managed in any way via the REST API. Global access to users will not be available, even to global administrators (super admin).

4.8

The users endpoint will be improved significantly for the 4.8 release, ideally providing full compatibility with how users are currently managed in a multisite configuration. The ticket for this task is #39544.

Here are the expectations for the users endpoint in 4.8:

  • POST to site.com/wp/v2/users/ with an existing global user’s email address adds the existing global user to a site.
  • POST to site.com/wp/v2/users/ with complete new user data creates a new global user.
  • GET to site.com/wp/v2/users/ with a global parameter set to true lists all global users.
  • GET to site.com/wp/v2/users/ without a global parameter lists only the site’s users.
  • GET to site.com/wp/v2/users/<id> with a global parameter set to true lists data for a global user, even if they are not a member of the site.
  • GET to site.com/wp/v2/users/<id> without a global parameter lists data for a user only if they are a member of the site.
  • PUT to site.com/wp/v2/users/<id> with a global parameter set to true updates a global user and data for a user that is in the global context.
  • PUT to site.com/wp/v2/users/<id> without a global parameter updates a data for a site user that is in the site context. This is how role data can be passed to maintain a user’s relationship with the site.
  • DELETE to site.com/wp/v2/users/<id> with a global parameter set to true deletes the user completely.
  • DELETE to site.com/wp/v2/users/<id> without a global parameter removes the user from the site.

Note that while users exist in a global context, data attached to these users can exist in the global context (e.g. email address) or in a site context (e.g. site role). There may need to be a method for registering user meta in a way that specifies whether it should be treated as global or site data.

Much of the work for 4.8 is still fluid. Please leave feedback on this approach in the comments and join office hours in #core-multisite on Tuesday 17:00 UTC!

#multisite, #networks-sites, #rest-api

Providing a REST API sites endpoint for multisite

One of the objectives for multisite is to determine how sites can be managed with the REST API. This has been an agenda item for the last two weeks and quite a bit has been processed. This will continue to be a topic, so please stop in for #core-multisite office hours on Tuesday 17:00 UTC and please leave your feedback in the comments on this post.

January 17 chat log and January 24 chat log in #core-multisite

Attendees: @kenshino, @vizkr, @danhgilmore, @iamfriendly, @flixos90, @schlessera, @sergeybiryukov, @pbearne, @paaljoachim, @streetlamp, @jnylen0, @loreleiaurora, @maximeculea

The requirements for the /sites/ endpoint can be summed up with these assertions:

  • The /sites/ endpoint should provide a useful set of data for each site without requiring the use of switch_to_blog().
  • It should be possible to query /sites/ for something other than ID, domain, and path.

In its current state, any /sites/ endpoint is limited to the fields in the wp_blogs table. Data such as a site’s name and description are stored in each individual site’s wp_#_options table.

Given a list of 20 sites, switch_to_blog() will be used 20 times so that get_option() can be used to retrieve things like home, siteurl, blogname, and blogdescription. An example of how inefficient this is can be found in the generation of the My Sites menu. Caching has gotten better with the introduction of WP_Site and WP_Site_Query, but there is an opportunity to change how the information is stored.

In #37923, @johnjamesjacoby suggests the introduction of a wp_blogmeta table that provides access to some site information in a common table. After discussing this in the January 17 chat, @iamfriendly and @flixos90 agreed to each take a look at core’s default options stored in wp_options and determine which made sense in a shared wp_blogmeta table. Richard’s results can be found in a comment on the ticket and Felix’s in the Slack discussion.

After some discussion in the January 24 chat, that list can be pared down a bit more.

  • home
  • siteurl
  • blogname
  • blogdescription
  • admin_email

The creation of a new table, wp_blogmeta, and migration of data for each site from wp_#_options is something that needs feedback. Without this change, a /sites/ endpoint is still possible, but may be limited. With the change, a /sites/ endpoint is much more useful, but requires a careful migration process.

#multisite, #networks-sites, #rest-api

Controlling access to REST API user functionality for multisite

Following last week’s discussion in #core-multisite (read the recap) this week’s office hours agenda was to continue the chat about the multisite-related enhancements for the REST API users endpoint, focussing heavily on how to access the required functionality. Here is a wrap-up of the discussion.

Chat log in #core-multisite

Attendees: @jeremyfelt, @jnylen, @nerrad, @ipstenu, @earnjam, @kenshino, @maximeculea, @mikelking, @lukecavanagh, @flixos90

Now that the way on how one should be able to modify user roles per site was clarified last week, this week the focus laid on where one should be able to perform those actions. The current state of the wp-json/wp/v2/users endpoint in multisite is:

  • The users overview accessible with a GET request to wp-json/wp/v2/users only lists users that are part of the current site.
  • When creating a user with a POST request to wp-json/wp/v2/users, that user is created and added to the current site. When providing the roles parameter, the passed roles are added to the user, otherwise the user will still be part of the site, but without any role. See #38526 for background.
  • It is possible to both read and edit any user from any site with a request to wp-json/wp/v2/users/<id>, regardless of whether the user is part of that site.
  • A DELETE request to wp-json/wp/v2/users/<id> results in an error. See #38962 for background.

After the discussion about how to be able to add a specific user to a site, update their site capabilities and remove them from a site, this week’s chat revolved around where these actions can be accessed, as they are for the most part network-specific actions not available to a site administrator. The approach that was agreed on is:

  • The users overview at wp-json/wp/v2/users should continue to only show users of that site by default, but a request like wp-json/wp/v2/users?global=true should show all users in the WordPress setup. This parameter must only be available to network administrators though, more specifically users with the manage_network_users capability. In the future a network parameter might also be introduced for support of multi networks, but at this point core does not support querying users per network. Accessing global users should be available from all sites in a setup instead of only from the main site. While this approach makes these endpoints duplicates of each other, it has several benefits like preventing the requirement for cross-domain requests, allowing easier API discovery and not requiring the main site of a setup to be exposed to REST API calls to a sub site.
  • Assigning an existing user to a site and removing a user from a site should generally be only available to network administrators, and the site administrators of the site that is being interacted with.
  • Similarly, editing a user that does not belong to the current site should only be possible for a network administrator. Currently this is available to site administrators as well which is probably wrong.
  • Deleting any user completely should only be available to a network administrator. A good way to handle the reassign parameter needs to be found though.

Before coming to the conclusion that dealing with multisite functionality at the existing users endpoint, the possibility of introducing a multisite-specific endpoint only available on the main site was discussed. However this was considered not practical due to the nature of how users work in WordPress. Having separate endpoints for other network-wide functionality might still be a possibility though as long as that component solely affects the network admin, so this idea is something to keep in mind for thought about further network functionality endpoints in the future.

Back to the users endpoint, one related question that came up is:

  • Should the sites a user belongs to be available at the wp-json/wp/v2/users endpoint or at a future wp-json/wp/v2/sites endpoint? If they were available in the wp-json/wp/v2/users endpoint, every user entity would have a new sites key available if the current user had sufficient permissions to see these. If they were available in the wp-json/wp/v2/sites endpoint, that endpoint could easily support this functionality through usage of a user parameter.

@jeremyfelt suggested to look at the “Add New User” screen in the site admin to have a good use-case for how to scaffold the multisite functionality of the API endpoint. This has helped during this week’s office-hours and can also be beneficial in the future. Eventually this screen should be revamped entirely, being powered by the REST API.

Regarding the enhancements of the users endpoint, a general ticket for this task was opened at #39544. This ticket is meant to be used for discussion on the topic, while separate smaller tickets should be opened for actually implementing the individual pieces. For now feedback is welcome on that ticket. The discussion on multisite improvements for the REST API will continue at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

 

 

#multisite, #networks-sites, #rest-api, #users

Improving the REST API users endpoint in multisite

One of the objectives for multisite is sorting how users are managed with the REST API. This was one of the agenda items for last week’s #core-multisite office hours and generated some good discussion. Here’s a wrap-up of the ideas and thoughts from that discussion.

Chat log in #core-multisite

Attendees: @iamfriendly, @johnjamesjacoby, @nerrad, @florian-tiar, @mikelking, @earnjam, @jeremyfelt

Users in multisite exist globally and are shared among sites on one or more networks. Users are associated with sites in the user meta table with a wp_#_capabilities key.

The current state of the wp-json/wp/v2/users endpoint for multisite is:

  • A POST request for a new global user to the main site creates the user and associates them with the main site only.
  • A POST request for a new global user to a sub site creates the user and associates them with the sub site only.
  • A POST request for an existing global user results in an error.
  • A PUT request for an existing global user to a sub site updates the user’s meta with a capability for that sub site.
  • A DELETE request on multisite is invalid and results in an error. See #38962.

It is not possible to remove a user from an individual site or to delete the user from the network.

Previous tickets: #38526, #39155, #38962, #39000

The following are a few thoughts expressed separately from the above summary.

  • The right way to associate existing objects over the REST API is with a PUT request.
  • The right way to disassociate existing objects is with a PUT request.
  • Linked previous discussion – “Deleting an item should always delete an item
  • We already have functions like remove_user_from_blog() and add_user_to_blog() available to us.
  • Does “add” invite or literally add? This can probably be included as data in the PUT request.
  • What happens if an API client is built for single site and then that site gets switched to multisite?
  • Handling bulk actions on an endpoint would be nice. (e.g. Add a user to multiple sites) No endpoint has implemented batch handling yet though.

Initial tasks:

  • It should be possible to remove a user from a site with a PUT request to the wp-json/wp/v2/users/# endpoint.
  • It should be possible to delete a global user with a DELETE request to the wp-json/wp/v2/users/# endpoint once all sites have been disassociated.

New tickets will be created soon for these tasks. Please leave any initial feedback in the comments on this post covering the assumptions and conclusions made above. There will be another round of discussion during tomorrow’s #core-multisite office hours at Tuesday 17:00 UTC.

/cc @rmccue and @kadamwhite

#multisite, #networks-sites, #rest-api, #users

Multisite Office Hours Recap (December 13, 2016)

Multisite office hours are held every Tuesday at 17:00 UTC in #core-multisite. The next will be Tuesday 17:00 UTC.

The weekly agenda was to discuss long-term goals for multisite and prioritize tickets to work on related to one of the three new focuses, particularly the REST API.

Today’s chat log

Attendees: @iamfriendly, @mista-flo, @johnjamesjacoby, @loreleiaurora, @johnbillion, @jeremyfelt, @flixos90

Chat Summary:

  • @jeremyfelt, @flixos90, @johnjamesjacoby, and @johnbillion sat down for a chat at WCUS to discuss multisite topics and possible future focuses. @jeremyfelt shared his notes from that session. Items of particular interest in today’s chat were:
    • Identify statistics for multisite usage, size of multisites: Possibly sending more general multisite-related metrics during core updates could be helpful and should be further discussed. In addition to whether multisite is enabled and the count of sites (on the main network) it could be helpful for future decisions to know how many sites there are in total, how many networks exist and whether user / site registration is open or not.
    • Implementation of network user roles: A ticket is in place at #39174. @flixos90 requested feedback: Everyone interested is asked to read through the ticket description and provide an elaborate response with how they think network roles should work. Then a discussion based on these approaches can start.
    • Network administrators should be able to enable themes on the site level: It is currently not very convenient for a network administrator to switch back to the network admin to enable a theme when currently working within the site admin.
    • Disable plugins per site in the network admin: Sometimes not all plugins should be visible on all networks or all sites, which is currently not possible. However, even for a disabled theme a network administrator should still be able to “silently” activate it on a site. Plugins would need to be enabled by default, so it would be the opposite way of how themes are currently handled. This also ensures there are no concerns with backward-compatibility. UI-wise an improved flow should be investigated instead of simply pursuing a similar approach to how enabling themes works: Having all the functionality inside the Network > Plugins screen would be helpful, with row and bulk actions for “Network Disable” as well as “Disable for Site…” with the latter having some kind of popover with an autocomplete field for sites. A related ticket to take into account for the implementation is #38652 which @johnbillion opened to introduce singular capabilities for managing plugins.
    • Whichever flow is agreed on for disabling plugins per site or network, once this functionality is implemented, it will probably make sense to “back-port” the behavior to improve the existing theme and maybe even user handling in a multisite.
  • At this point, the highest priority is multisite support for the REST API.
    • The users endpoint should be improved, particularly the implementation of DELETE for a user on a multisite as well as removing a user from a site or setting their capabilities per site via PUT (see #39000 and #38962 for related tickets).
    • A sites endpoint should be added.
    • A networks endpoint is not as important and should for now only live in a standalone plugin if there is interest to implement it.
  • A general question raised was how No-JS fallbacks for REST API functionality in the admin should be handled. Should JavaScript become a requirement to use such new functionality or should the REST API only be used to improve existing PHP-only features? These questions should be discussed in a broader scope than just multisite since they affect all of WP Admin.
  • Possible usage for a REST API sites endpoint:
    • Replacing “My Sites” in the toolbar with a menu that doesn’t load until needed and pulls all sites from the API.
    • The possible autocomplete field for disabling plugins for a specific site (see above) could pull sites from the API.
  • @mista-flo brought up #13743 and asked for feedback. Everyone in the chat agreed that the ability to set a default theme for a network would be a valuable enhancement over the existing constant. For the approach, introducing a new row action “Set default theme” the Network > Themes list table was suggested, alongside an indicator “Default Theme” in that same table. This is an enhancement that can be worked on beside the other important REST API features.
  • A Google document was opened a while ago to gather input for a future multisite roadmap, as a follow-up to 2013’s post. @jeremyfelt highlighted that this document should continue to move forward in order to hopefully be able to publish an actual roadmap in early January.

#4-8, #multisite, #network-sites

Multisite Focused Changes in 4.7

Howdy. The 4.7 release cycle has been a chance to build on some of the work from the last couple releases in multisite.  If you’d like more detail, check out the full list of multisite focused changes in this release.

get_blog_details() replaced with get_site()

A lot of progress has been made over the last few releases to get things in place for this transition. Now that WP_Site and WP_Network objects exist and are accessible with functions like get_site() and get_network(), they can be implemented throughout core.

In WordPress 4.7, get_blog_details() was replaced throughout core code with the modern  get_site(). The roadmap for this includes deprecating get_blog_details() in WordPress 4.8, so take this cycle as a chance to move your code in that direction.

get_site() is often a direct replacement, though get_sites() can also be used to query for sites when an ID is not available.

See #37102 for details on this change.

blog_details filter deprecated

In combination with the decision to stop using get_blog_details() throughout core, the (not widely used) blog_details filter has been deprecated. It has been added to get_site() to provide backward compatibility with the above change and will fire with a deprecation notice. Plugin code should use the site_details filter instead. See #38491 for details on this change.

_network_option actions and filters get $network_id

The $network_id associated with the use of a _network_option() function is now passed to the filters and actions that fire within. This provides granular control that was not available when first introduced. See #38319, #38320, #38321, and #38322 for details on this change.

wp_get_network() deprecated

It is now recommended that get_network() is used instead. See #37553 for this change.

 

 

#4-7, #dev-notes, #multisite, #network-sites

Upcoming Multisite Bug Scrubs

After the initial multisite bug scrub in the 4.7 release cycle last week, we will continue going through the list of bug tickets this week on Thursday 20:00 UTC, in #core-multisite as usual. The respective report was about bug tickets with the multisite focus that were opened within the last year. We have 8 tickets to go for that report, so we should be able to finish it this week.

After that, we will continue having regular multisite bug scrubs that are held every Thursday at 20:00 UTC, then focussing on multisite tickets milestoned for the 4.7 release.

If you aren’t available during the weekly multisite bug scrubs, feel free to leave a comment here or in #core-multisite if you have other tickets you’d like to see covered.

See you then!

#4-7, #bug-scrub, #multisite