Multisite Recap for the week of November 6th

Office Hours Recap

The agenda for this office hours meeting was open floor.

The meeting's chat log

Attendees: @florian-tiar, @spacedmonkey, @desrosj, @flixos90, @mikelking, @jeremyfelt

Chat Summary:

  • There was a loose discussion on the tickets that should be focused on at the beginning of the 5.0 cycle.
  • Caching in WP_Site_Query will be a big focus at first. Tickets #42252, #42280, #42251. There was a brief discussion on some approaches.
  • Implementing new install and update methods in #41333.
  • Using the meta API for network options in #37181.
  • The new multisite roadmap is getting there and should be ready to publish before WordCamp US.

Next meeting

The next office hours will take place on November 14th, 2017, 17:00 UTC. Its agenda will be:

  • Discuss adopting Trello as a way to manage status, progress, and ownership on tasks.
  • Progress on multisite roadmap with a target to publish an initial version before WCUS.
  • Any progress on 5.0 tickets.

#4-9, #multisite, #network-sites, #summary

Multisite Recap for the week of October 2nd

Office Hours Recap

The agenda for this office hours meeting was to prepare remaining open enhancements for the 4.9 release.

The meeting’s chat log

Attendees: @flixos90, @spacedmonkey, @jeremyfelt

Chat Summary:

  • #40228 – Review the latest patches for using get_site_by() in get_blog_details().
  • #40201 – Review the latest patches for using get_site() in clean_blog_cache(). @flixos90 and @jeremyfelt agreed that requiring a site ID is okay for clean_blog_cache(), moving the refresh_blog_details hook into clean_blog_cache() is appropriate, and deprecating the refresh_blog_details hook in favor of clean_site_cache is good.
  • @flixos90 committed to committing #40201 and #40228 before 4.9 beta.
  • #42072 and #42073 – Discussed both of these, which involve limiting get_sites() calls to 1 result when only 1 result is used. @jeremyfelt took ownership of these for 4.9.
  • #41762 – Agreed to use WP_Network_Query in ms_load_current_site_and_network(), but to keep the current-network cache key for now.

Next meeting

The next office hours will take place on October 4, 2017, 16:00 UTC. Its agenda will be to go over remaining bugs for 4.9 and the progress of this cycle’s dev notes.

Ticket Scrub Recap

The agenda for this ticket scrub was to review remaining enhancements for the 4.9 release.

The meeting’s chat log

Attendees: @ramiy, @sergey, @afragen, @flixos90, @spacedmonkey, @jeremyfelt, @johnjamesjacoby, @desrosj

Chat Summary:

  • #40180 – Discussion on some of the remaining details for get_site_by().
  • #40228 – Agreed to finish and commit #40180 so that the patches for get_site_by() in get_blog_details() would be easier to review.
  • #40201 – Discussion if clean_blog_cache() should support an empty argument as a replacement for refresh_blog_details(). Agreement that it’s okay to continue requiring that a site be specified.
  • Discussion on when or whether to deprecate refresh_blog_details() or its hook refresh_blog_details, which is now a part of clean_blog_cache(). It seems that it’s okay to postpone this until get_site_by() and clean_blog_cache() are more widely used.

Next meeting

@jeremfelt did not clarify early enough whether or not a ticket scrub will take place on October 3, 2017, 17:00 UTC. 🙈 A loose agenda would be to cover remaining bugs for the 4.9 cycle.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #network-sites, #summary

Multisite Agenda for the week of September 4th

Office Hours Agenda

This is the agenda for the weekly office hours meeting on Tuesday 16:00 UTC in #core-multisite.

  • Open floor for tickets.
  • Review progress on roadmap and related discussion.

Ticket Scrub Agenda

There will be no ticket scrub this week in #core-multisite. An agenda for next week’s ticket scrub will be posted later this week.

Please join the chat if you’re interested in one of the topics. In case you cannot make the respective meeting, we will be working on publishing a recap post afterwards. If you have some thoughts beforehand or would like something related to be part of the agenda, feel free to share your ideas in the comments for this post. See you in the chat!

#4-9, #agenda, #multisite, #network-sites

Multisite Recap for the Week of August 28th

Office Hours Recap

The agenda for this office hours meeting was to take a look at progress on the multisite roadmap and to continue discussing the best approach to checking for the existence of a wp_blogmeta database table (See #37923).

The meeting’s chat log

Attendees: @flixos90, @desrosj, @stephdau, @dac, @johnbillion, @spacedmonkey, @jeremyfelt

Chat Summary:

  • During the core dev chat, everyone was open to the roadmap being published at make.wordpress.org/core/roadmaps/multisite. The REST API team is also interested in posting a roadmap under that structure.
  • Quite a bit of progress has been made by @flixos90 so far on the multisite roadmap document.
  • A good way to contribute to the roadmap now is to read through critically and make comments/suggestions on the direction and content.
  • Everyone is free to make changes/comments. Ping @jeremyfelt if you would like access.
  • @johnbillion is going to add some items related to multisite administration.
  • More needs to be built out around multisite’s bootstrap and the future of “global context” above the level of network.
  • There was some discussion on what an objective on the roadmap for multisite’s bootstrap may be. Everyone seems to agree that “increasing stability and testability” of the bootstrap is a good objective.
  • There should be a general section at the end that covers some things that don’t necessarily fit as part of the main roadmap at this time. That can be considered “a summary to encourage discussion/contribution even if what you want isn’t on the roadmap.”
  • Quite a bit of ground was covered on #37923 as to whether the existence of a wp_blogmeta table (after upgrade) should be stored as a main network option, as an option on every network, or in a new place entirely.
  • @spacedmonkey brought up concerns about the performance issues of loading an option from a different network. During multisite’s bootstrap, all of the current network’s options are loaded into memory. Storing in a single place would result in an extra DB query/cache lookup.
  • @flixos90 and @jeremyfelt prefer (so far) storing the data as an option on the main network. The idea that a global setting storage may exist one day makes it tempting to treat this data as global now.
  • A next step is to lookup more possible “global context” data that could be stored. @spacedmonkey brought up global terms and ms-files as two immediate options. Brainstorming should continue via chat and on the ticket (#37923).

The next office hours will take place on Tuesday 16:00 UTC. An agenda for it will be posted in advance.

Ticket Scrub Recap

The agenda for this ticket scrub was to review 4.9 ticket progress.

The meeting’s chat log

Attendees: @flixos90, @desrosj, @sergey, @paaljoachim, @jeremyfelt, @spacedmonkey

Chat Summary:

  • #29684 – Had a lengthy discussion on the possibilities of this ticket. Chatted through whether a networks main site ID should be stored as a network option. For the time being, let’s stick with not storing the data anywhere.

There will be no ticket scrub Monday 17:00 UTC. A schedule and agenda for the next ticket scrub will be posted next week.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #network-sites, #summary

Multisite Recap for the week of August 14th

Office Hours Recap

The agenda for this office hours meeting was to discuss the format of the multisite roadmap that will be compiled from many of our recent conversations.

The meeting’s chat log

Attendees: @desrosj, @flixos90, @spacedmonkey, @vizkr, @jeremyfelt

Chat Summary:

  • An initial multisite roadmap document has been started. If anyone would like access to edit, please ping @jeremyfelt on Slack.
  • The previous multisite roadmap (2013) was verbose. @jeremyfelt mentioned that it provided the thought process behind a lot of things that were going to become decisions at some point.
  • That the previous roadmap was posted on make/core and then lost in the timeline makes discoverability difficult.
  • There should be a specific URL where the roadmap can be published and evolve over time. Additional (verbose) updates on progress or new initiatives can be posted as posts on make/core and then linked from the roadmap as needed.
  • During the weekly core dev chat, we’ll propose a make.wordpress.org/core/roadmap/ structure under which make.wordpress.org/core/roadmap/multisite/ can exist as well as pages for other components or focuses.
  • It’s also possible we can follow the customizer component’s lead and publish under https://make.wordpress.org/core/components/networks-sites/. Note that this only covers one component rather than the focus.
  • A ticket should be created for everything we plan so that people can help when available.
  • @flixos90 is going to spend some time working on the roadmap document some more to get it closer to the format that we’ve been discussing.

The next office hours will take place on Tuesday 16:00 UTC. An agenda for it will be posted in advance.

Ticket Scrub Recap

The agenda for this ticket scrub was to discuss approaches for better clarifying site statuses, and then have an open floor for further tickets.

The meeting’s chat log

Attendees: @flixos90, @sergey, @ina2n, @afragen, @pmbaldha, @paaljoachim, @desrosj, @spacedmonkey, @jeremyfelt

Chat Summary:

  • #17164 – More elegant handling of site ‘archive’ options for Multisite
  • #39158 – Unify site deactivation process
  • #15801 – Network Admin: Deactivated / Deleted inconsistency
  • These 3 tickets are all related in some way, though deal with different issues around site statuses.
  • The checkboxes at wp-admin/network/site-info.php don’t convey any real information about their meaning or a workflow.
  • Different terms are used in the MS sites list table for network admins (deactivate) and in the site admin (delete).
  • Different error messages are displayed on the front end when a site is archived or deactivated. There should be better documentation for what these mean in the screen help.
  • @flixos90 – “Deactivating should be called deactivating everywhere”, “we should only use the term Delete when it really is delete”
  • It seems that #39158 is a ticket about workflow and unification of deactivate/delete. #15801 is a ticket about terminology.
  • Priorities via @flixos90:
    • change Deleted to Deactivated where it’s not actually Deleted
    • Figure out what all these actions do exactly.
    • Document what these actions do
  • We need to support these workflows in the sites endpoint.
  • #40736 – Ensure that get_blog_count() and get_user_count() return an integer. Specifically—Can the return type be safely changed without issue and/or should we deprecate get_blog_count() in favor of get_site_count(). We aren’t too worried about changing the return type as this function doesn’t appear to be used much outside of core.
  • #41344 – Secure Email Integration. We agreed to look at this ticket next week.
  • #40764 – Multisite theme update not showing new version number. This ticket came up right at the end and can be covered next week.

The next ticket scrub will take place on Monday 17:00 UTC. An agenda for it will be posted in advance.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #networks-sites, #summary

Multisite Recap for the week of July 31st

Office Hours Recap

The agenda for this office hours meeting included the discussion of a “global” state for multisite.

The meeting’s chat log

Attendees: @desrosj, @flixos90, @paaljoachim, @johnbillion, @feshin, @spacedmonkey, @stephdau, @jeremyfelt

Chat Summary:

  • In the multisite context, “global” is the hierarchy level above networks. A global installation of WordPress has one or more networks. Each of these networks has one or more sites.
  • Users are global in that wp_users provides the same set of users used across all sites on all networks. There is no concept of a global role other than super admin. Another level could be created so that a global admin controls all networks and network admins control individual networks.
  • Two important pieces of this discussion are global storage, data about the configuration of the whole, and global roles, data about global users’ relationships with sites.
  • Items that fit under global storage could be a total user count, a total site count, a network count, globally active plugins, globally enabled themes, update checks.
  • @jeremyfelt showed a custom UI for activating and deactivating global plugins. @flixos90 mentioned his global admin plugin as a UI example. It seems like these are good examples of how core can provide APIs to make interfaces easier to build. We should not necessarily be focused on trying to build the UI.
  • There are two options for storing global data — as network ID 0 in wp_sitemeta or as a new global table. After some discussion it seems that using network ID 0 is the best answer. As part of an effort to use the meta API for managing network meta (see #37181), the API will need to be updated to support an object ID of 0. It may be that this is only necessary for network meta, but it’s possible other components could find it useful.
  • The current plan is to implement global multisite storage in wp_sitemeta with a network ID of 0. This is relatively low priority compared to other efforts, but implementation itself will not be too involved. A lot of testing for breakage must be done. As @stephdau mentions, “we all know how much room there is for error with 0 values in data”. 🙂

The next office hours will take place on Tuesday 16:00 UTC. An agenda for it will be posted in advance.

Ticket Scrub Recap

The agenda for this ticket scrub was to continue reviewing multisite tickets assigned to the 4.9 milestone.

The meeting’s chat log

Attendees: @desrosj, @drewapicture, @flixos90, @florian-tiar, @jmdodd, @sergey, @spacedmonkey

Chat Summary:

  • A third opinion would be appreciated regarding the discussion around #29684 about whether to store the main site of a network in a network option. @spacedmonkey argues that it would provide an easier lookup, while @flixos90 argues that it would be redundant and unreliable because of existing sites not having it and the new get_main_site_id() would be just as easy-to-use.
  • @sergey is going to review #41285 to come to a conclusion whether removing the unused $site_id and $public globals in the multisite bootstrap process would have any bad side effects.
  • #38645 and #36961 are closely related as they both deal with switching context, the first for roles, the latter for user capabilities. Their methods and properties used for that should be similar, for a better DUX. @spacedmonkey pointed out that eventually the new blogmeta table would be helpful with this, however that shouldn’t be the focus of these tickets. At the point of this writing, @flixos90 has updated patches on these that require a review.

The next ticket scrub will take place on Monday 17:00 UTC. An agenda for it will be posted in advance.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #networks-sites, #summary

Multisite Focused Changes in 4.8

Here’s an overview of the developer facing changes made in multisite for the 4.8 cycle. If you’re interested in more detail, checkout the full list of tickets.

New capabilities

As part of the goal to remove all calls to is_super_admin() in favor of capability checks, two new meta capabilities have been introduced in this release:

  • upgrade_network is used to determine whether a user can access the Network Upgrade page in the network admin. Related to this, the capability is also checked to determine whether to show the notice that a network upgrade is required. The capability is not mapped, so it is only granted to network administrators. See #39205 for background discussion.
  • setup_network is used to determine whether a user can setup multisite, i.e. access the Network Setup page. Before setting up a multisite, the capability is mapped to the manage_options capability, so that it is granted to administrators. Once multisite is setup, it is mapped to manage_network_options, so that it is granted to network administrators. See #39206 for background discussion.

Both capabilities work in a backward-compatible fashion, so no changes should be required in plugins that deal with those areas. The capabilities however allow more fine-grained control about access to their specific areas. As an additional reminder: Usage of is_super_admin(), while not throwing a notice, is discouraged. If you need to check whether a network administrator has access to specific functionality in your plugin, use one of the existing network capabilities or a custom capability. See #37616 for the ongoing task.

New Hooks

  • deleted_blog fires after a site has been deleted. See #25584
  • signup_site_meta filters site meta in wpmu_signup_blog(). See #39223
  • signup_user_meta filters user meta in wpmu_signup_user(). See #39223
  • minimum_site_name_length filters the minimum number of characters that can be used for new site signups. See #39676

Network-specific site and user count control

With WordPress 4.8, several functions related to network site and user counts now support an additional $network_id parameter that allows to read and update those counts on a different network than the current one. While this mostly increases general flexibility, the new parameter can be leveraged to fix inconsistencies, for example prior to 4.8 the creation of a new site on another network would falsely trigger a recalculation of the current network instead of the network being actually modified. See #38699 for the bug ticket and resulting task.

The functions that now support a $network_id parameter are:

  • get_blog_count(). See #37865.
  • get_user_count(). See #37866.
  • wp_update_network_site_counts(). See #37528.
  • wp_update_network_user_counts(). See #40349.
  • wp_update_network_counts(). See #40386.
  • wp_maybe_update_network_site_counts(). See #40384.
  • wp_maybe_update_network_user_counts(). See #40385.
  • wp_is_large_network(). The filter wp_is_large_network now passes the network ID as fourth parameter as well. See #40489.

#4-8, #dev-notes, #multisite

Approaches for a complete sites endpoint and an expanded WP_Site_Query

Multisite office hours are held every Tuesday at 16:00 UTC in #core-multisite. The next will be Tuesday 16:00 UTC.

Creating a useful sites/ endpoint for the REST API and making WP_Site_Query more useful have been frequent topics over the last few weeks in #core-multisite. Quite a bit of discussion has been centered around the idea of a global wp_blogmeta table, whether it’s a good fit for core, and (if so) how to approach its introduction. See #37923 and the previous make/core post discussing a sites endpoint for additional background.

The intent of this post is to step back a bit and clarify the issues at hand to help identify the right solution(s).

The initial site information problem

A request to a global wp-json/wp/v2/sites/ endpoint on a network should return a set of site objects, similar to how wp-json/wp/v2/posts/ returns a set of post objects. A request to wp-json/wp/v2/sites/4 would return a single site object.

Written today each site object, represented as WP_Site in PHP, would provide at least this data:

  • ID
  • domain
  • path
  • registered
  • last_updated
  • public
  • archived
  • mature
  • spam
  • deleted
  • lang_id

The above fields all mirror what is available in the global wp_blogs database table installed with multisite. These are useful on their own, but additional data is frequently required.

Example: One piece of the admin that would be great to power with the REST API is the My Sites menu that multisite users see in the toolbar. To build this view, the home URL, admin URL, and site name are all necessary. In PHP, this data is made available through magic properties on the WP_Site object. When home, siteurl, blogname, or post_count is requested, the site uses switch_to_blog() and get_option() to populate the data from the individual site’s options table. If 25 sites are on the list, this generates 25 context switches and accesses 25 different tables.

There are at least a few approaches here:

  • Mirror the existing PHP experience and ensure properties are populated before the REST API response is returned. Possibly introduce a lighter weight switch_to_blog() option.
  • Provide a basic site object as well as an option to _embed other data in the response.
  • Migrate these properties into wp_blogs from wp_#_options as additional columns.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

The querying sites problem

It’s currently possible to query for sites by the default fields listed above. This data is useful for querying, but it would also be nice to query by a site’s name, description, or other piece of data at a global level.

Example: In the list table that displays all the sites of a network, it’s possible to query by a site’s domain or path, but not by it’s actual name. In a large network, a user may find it difficult to find a site when many sites share similar domains or paths. The user may know the site’s title, but not the address itself.

There is no real workaround for this issue in core right now as each site’s name is stored individually in that site’s wp_#_options table and cannot be queried collectively.

Possible approaches:

  • Migrate these properties into wp_blogs() from wp_#_options as additional fields.
  • Migrate these properties into a global wp_blogmeta table.
  • Migrate these properties into another shared global space.

Feedback please

Please leave any thoughts you have on possible approaches to these 2 problems. It would be especially helpful to identify some use cases that may not yet be clear, or approaches that are not listed above. All feedback is welcome. 🙂

#multisite, #rest-api

Improving the REST API users endpoint for multisite in 4.7.3 and 4.8

Multisite office hours are held every Tuesday at 17:00 UTC in #core-multisite. The next will be Tuesday 17:00 UTC.

Improving the users endpoint was the main focus of this week’s office hours. The following is a recap of the decisions from that discussion. Please leave your feedback in the comments on this post. Related meeting notes are available from the January 10th office hours and the January 3rd office hours.

Chat log

Attendees: @iamfriendly, @sergeybiryukov, @flixos90, @ssstofff, @vizkr, @jnylen0, @nerrad, @johnbillion, @jeremyfelt

4.7.3

Some small changes to the users endpoint for multisite should be made in 4.7.3. The ticket for these changes is #39701.

  • Fail when a GET request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.
  • Fail when a PUT request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

The expectation for the users endpoint in 4.7.3 is that only users from the current site can be listed or managed in any way via the REST API. Global access to users will not be available, even to global administrators (super admin).

4.8

The users endpoint will be improved significantly for the 4.8 release, ideally providing full compatibility with how users are currently managed in a multisite configuration. The ticket for this task is #39544.

Here are the expectations for the users endpoint in 4.8:

  • POST to site.com/wp/v2/users/ with an existing global user’s email address adds the existing global user to a site.
  • POST to site.com/wp/v2/users/ with complete new user data creates a new global user.
  • GET to site.com/wp/v2/users/ with a global parameter set to true lists all global users.
  • GET to site.com/wp/v2/users/ without a global parameter lists only the site’s users.
  • GET to site.com/wp/v2/users/<id> with a global parameter set to true lists data for a global user, even if they are not a member of the site.
  • GET to site.com/wp/v2/users/<id> without a global parameter lists data for a user only if they are a member of the site.
  • PUT to site.com/wp/v2/users/<id> with a global parameter set to true updates a global user and data for a user that is in the global context.
  • PUT to site.com/wp/v2/users/<id> without a global parameter updates a data for a site user that is in the site context. This is how role data can be passed to maintain a user’s relationship with the site.
  • DELETE to site.com/wp/v2/users/<id> with a global parameter set to true deletes the user completely.
  • DELETE to site.com/wp/v2/users/<id> without a global parameter removes the user from the site.

Note that while users exist in a global context, data attached to these users can exist in the global context (e.g. email address) or in a site context (e.g. site role). There may need to be a method for registering user meta in a way that specifies whether it should be treated as global or site data.

Much of the work for 4.8 is still fluid. Please leave feedback on this approach in the comments and join office hours in #core-multisite on Tuesday 17:00 UTC!

#multisite, #networks-sites, #rest-api

Providing a REST API sites endpoint for multisite

One of the objectives for multisite is to determine how sites can be managed with the REST API. This has been an agenda item for the last two weeks and quite a bit has been processed. This will continue to be a topic, so please stop in for #core-multisite office hours on Tuesday 17:00 UTC and please leave your feedback in the comments on this post.

January 17 chat log and January 24 chat log in #core-multisite

Attendees: @kenshino, @vizkr, @danhgilmore, @iamfriendly, @flixos90, @schlessera, @sergeybiryukov, @pbearne, @paaljoachim, @streetlamp, @jnylen0, @loreleiaurora, @maximeculea

The requirements for the /sites/ endpoint can be summed up with these assertions:

  • The /sites/ endpoint should provide a useful set of data for each site without requiring the use of switch_to_blog().
  • It should be possible to query /sites/ for something other than ID, domain, and path.

In its current state, any /sites/ endpoint is limited to the fields in the wp_blogs table. Data such as a site’s name and description are stored in each individual site’s wp_#_options table.

Given a list of 20 sites, switch_to_blog() will be used 20 times so that get_option() can be used to retrieve things like home, siteurl, blogname, and blogdescription. An example of how inefficient this is can be found in the generation of the My Sites menu. Caching has gotten better with the introduction of WP_Site and WP_Site_Query, but there is an opportunity to change how the information is stored.

In #37923, @johnjamesjacoby suggests the introduction of a wp_blogmeta table that provides access to some site information in a common table. After discussing this in the January 17 chat, @iamfriendly and @flixos90 agreed to each take a look at core’s default options stored in wp_options and determine which made sense in a shared wp_blogmeta table. Richard’s results can be found in a comment on the ticket and Felix’s in the Slack discussion.

After some discussion in the January 24 chat, that list can be pared down a bit more.

  • home
  • siteurl
  • blogname
  • blogdescription
  • admin_email

The creation of a new table, wp_blogmeta, and migration of data for each site from wp_#_options is something that needs feedback. Without this change, a /sites/ endpoint is still possible, but may be limited. With the change, a /sites/ endpoint is much more useful, but requires a careful migration process.

#multisite, #networks-sites, #rest-api