New Network and Sites Query Filters

WordPress 5.2 introduces new short circuit filters to WP_Site_Query and WP_Network_Query.

These two filters, sites_pre_query and network_pre_query, were introduced in [44983] and run before the database queries are executed. This enables short-circuiting the database query entirely to return custom results. Returning a non-null value from either filter will bypass WordPress’s default network and sites queries respectively (similar to the users_pre_query filter introduced in #44373).

Sites Query Filter

Developers should note that filtering functions that require pagination information are encouraged to set the found_sites property of the WP_Site_Query object (which is passed to the filter by reference). If WP_Site_Query does not perform a database query, it will not have enough information to generate these values itself.

Using the Filter

Below is a rough example of how a plugin can use the filter to replace the default behavior of WP_Site_Query with a call to a remote data store.

function myplugin_do_external_site_query( $sites, $site_query ) {
	$response = wp_remote_get( 'https://my-remote-data-store/foo/bar' );

	if ( 200 === wp_remote_response_code( $response ) ) {
		$response           = json_decode( wp_remote_retrieve_body( $response ) );
		$sites              = array_map( 'intval', $response->site_ids );
		$query->found_sites = (int) $response->found_sites;
	}

	return $sites;
}
add_filter( 'sites_pre_query', 'myplugin_do_external_site_query', 10, 2 );

Networks Query Filter

Developers should note that filtering functions that require pagination information are encouraged to set the total_networks property of the WP_Network_Query object (which is passed to the filter by reference). If WP_Network_Query does not perform a database query, it will not have enough information to generate these values itself.

Using the Filter

Below is a rough example of how a plugin can use the filter to replace the default behavior of WP_Network_Query with a call to a remote data store.

function myplugin_do_external_network_query( $networks, $network_query ) {
	$response = wp_remote_get( 'https://my-remote-data-store/foo/bar' );

	if ( 200 === wp_remote_response_code( $response ) ) {
		$response              = json_decode( wp_remote_retrieve_body( $response ) );
		$networks              = array_map( 'intval', $response->network_ids );
		$query->total_networks = (int) $response->total_networks;
	}

	return $networks;
}
add_filter( 'network_pre_query', 'myplugin_do_external_network_query', 10, 2 );

Other Similar Query Filters

Similar filters for WP_User_Query and count_users() were added in WordPress 5.1.

Several additional filters for similar query objects are currently being explored and worked on, and are currently slated for a future release:

  • A short circuit for WP_Comment_Query (#45800).
  • A short circuit for WP_Term_Query with a plan to add a terms_pre_query filter (#41246).

Why Add These Filters?

These query pre-filters enable plugins to use an alternate database store for queries, for example returning results from an external service such as an Elasticsearch server. The process started with the main post query, and these are now being expanded that to other queries in WordPress.

#5-2, #dev-notes, #multisite, #networks-sites

Multisite Support for Site Metadata in 5.1

WordPress multisite introduces a new database table to store metadata associated with sites. This allows for the storage of arbitrary site data relevant in a multisite / network context.

Site metadata provides an alternative to using options and can be retrieved from multiple sites in a more performant way—without calling switch_to_blog(). Sites can now also be queried by their meta with parameters supported by WP_Meta_Query.

The new wp_blogmeta table is a global table in WordPress and developers should be cautious not to overuse it. There is a trade-off between using site options and site metadata, so it is recommended to think about every piece of extra data associated with a site and how it should be stored.

  • Use options when the data concerns only the site itself. Using an option should continue to be the default approach for site data.
  • Use site metadata when the data also needs to be heavily used in the network context, especially when sites need to be queried by it.

A network update is required to install the new database table. The site meta API includes a function is_site_meta_supported() to ensure no unexpected errors occur when that process has not run yet. As of WordPress 5.1, there is only a single use-case for the site meta in core which regards the fatal error protection mechanism (see #44458).

This is how the API can be used:

  • get_site_meta( $id, $meta_key, $single )
  • update_site_meta( $id, $meta_key, $meta_value, $prev_value )
  • add_site_meta( $id, $meta_key, $meta_value, $unique )
  • delete_site_meta( $id, $meta_key, $meta_value )

All of these functions are only available in multisite, however they work similarly to other metadata wrapper functions, such as for posts, terms, comments and users. In addition to these functions, it is now possible to use the common meta query arguments when querying sites with WP_Site_Query or its wrapper get_sites().

Note that the database table for site metadata is called wp_blogmeta. This is because wp_sitemeta is already used for another database table, and furthermore the new name works consistently alongside the related wp_blogs table that stores the most essential site data. The API exposed and described here uses the correct term “site”.

For background information on these changes, see #37923 and #40229.

An additional under-the-hood change related to this is that the metadata API is now loaded earlier in the WordPress bootstrap process so that it is available as early as the multisite bootstrap code. See #40948 for more information.

Props @jeremyfelt for review.

#5-1, #dev-notes, #multisite, #networks-sites

Multisite agenda: December 18

This is the agenda for the weekly office hours meeting on Tuesday, December 18th, 2018 at 17:00 UTC in #core-multisite.

As mentioned in #core-multisite last week, it’s been a while since the last organized multisite office hours. Let’s use this as an opportunity to regroup and determine what’s next in 2019. In particular, these topics will be covered:

  • Status of progress made in 2018. Nothing has shipped officially yet, but a full recap of all changes currently in trunk needs to be written so that the current state can be communicated.
  • Expectations for 2019. What reasonably matches (and doesn’t match) the overall WordPress project focus for the next year.
  • Office hours in 2019. What time, how frequently, who is leading them.

In addition to those general topics, @flixos90 would like to reserve 10-20 minutes to chat about the multisite integrations of WSOD protection in Servehappy. The core ticket is #44458 and patches are also applied as a pull request on GitHub.

Please join the chat if you’re interested in one of the topics. In case you cannot make the respective meeting, we will be working on publishing a recap post afterwards. If you have some thoughts beforehand or would like something related to be part of the agenda, feel free to share your ideas in the comments for this post. See you in the chat!

#agenda, #multisite, #networks-sites

Multisite Recap for the week of November 13th

Office Hours Recap

The agenda for this office hours meeting was to discuss adopting Trello as a way to manage status, progress, and ownership on tasks, as well as progress on multisite roadmap with a target to publish an initial version before WCUS.

The meeting’s chat log

Attendees: @desrosj, @flixos90, @jeremyfelt, @spacedmonkey, @vizkr

Chat Summary:

  • The full roadmap draft should be in place by November 21st, in order for the document to be published a week later, on November 28th, so that it is out before WordCamp US. @flixos90 and @jeremyfelt will have another look to complete the remaining sections.
  • After discussing possible usage of Trello in a collaborative way, the conclusion was that what is missing the most is an easy-to-use UI to manage required tasks and assignees for tickets on Trac. However most of the functionality is already in place and could be used through workarounds, so the decision was made not to use Trello for now. Instead the goal is to be more precise with defining tasks when assigning a ticket and generally assign tickets around between the responsible parties for a part of it more deliberately, instead of having a single owner through most of a ticket’s lifecycle. In a perfect world, there would be more possibilities to choose from when assigning a ticket to somebody, such as “update patch” or “write unit tests”, but for now these tasks can also be determined in regular comment text.

Next meeting

The next office hours will take place on November 21th, 2017, 17:00 UTC. Its agenda will be to review the roadmap draft and then discuss #42252.

Ticket Scrub Recap

No ticket scrub took place this week.

Next meeting

The next ticket scrub will take place on November 20th, 2017, 18:00 UTC. Its agenda will be to determine the current state and required tasks of tickets scheduled for 5.0 and assign them accordingly.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#multisite, #networks-sites, #summary

Multisite Focused Changes in 4.9

Here’s an overview of the developer facing changes made in multisite for the 4.9 cycle. If you’re interested in more detail, checkout the full list of tickets.

clean_blog_cache() replaces refresh_blog_details()

Since 3.5, refresh_blog_details(), which accepts a site ID, has been a wrapper of the clean_blog_cache() function, which requires a site object.

In WordPress 4.9, clean_blog_cache() has been adjusted to also accept a site ID and to invalidate caches for a deleted site in the same way. From now on clean_blog_cache() should be used instead of refresh_blog_details() which will be deprecated in a future release.

More importantly, the refresh_blog_details action has been deprecated in favor of the clean_site_cache action. See #40201.

New function get_main_site_id()

The WP_Network class has historically contained a $blog_id property indicating the ID of the main site of that network. However, since this property was never part of the wp_site database table, it is set manually in the multisite bootstrapping process. This results in it only being set for the current network. For any other network, code like get_network( $id )->blog_id would return 0.

The new get_main_site_id() function introduced in 4.9 provides the site ID of any network in an easy way. The function accepts an optional $network_id parameter, which defaults to the current network. Furthermore the magic property logic in WP_Network has been adjusted so that the $blog_id property (and its magic $site_id equivalent) is automatically set when requested. This ensures get_network( $id )->blog_id will always return a meaningful value. See #29684.

Refactored user capability and role switching

Switching the available roles and the current user’s capabilities no longer happens in switch_to_blog() and restore_current_blog(). Instead it has been moved to a new function, wp_switch_roles_and_user(), which is hooked into the site switching process. This provides a performance improvement by temporarily unhooking the function in cases where roles and capabilities do not need to be switched.

In addition, the available user roles are now correctly switched when switching sites, with refactored behavior in the WP_User and WP_Roles classes making this possible. These changes are more closely explained in the 4.9 post about role and capability improvements. For related tickets, see #36961 and #38645.

Site administrators can edit user roles through the REST API

While site administrators cannot edit user details in multisite, they are able to modify a user’s roles. In WordPress 4.9 this can now be achieved through the REST API by making a request such as PUT wp/v2/users/<id> and passing only the roles argument in the request body. No other arguments must be given as those would require the current user to have network administrator capabilities. See #40263.

Other Notes

  • The new can_add_user_to_blog filter can be used to prevent a user from adding specific users to a site or with a specific role. See #41101.
  • The old site network admin email address gets notified of a change to the address. See related security improvements for 4.9 and #39117.

#4-9, #dev-notes, #multisite, #networks-sites

Multisite Recap for the week of October 9th

Office Hours Recap

The agenda for this office hours meeting was to review 4.9 bug and task tickets that still need to be finished and merged within Beta, particularly continuing from where the ticket scrub meeting stopped.

The meeting’s chat log

Attendees: @earnjam, @flixos90, @jeremyfelt, @jjj, @johnbillion, @josheby, @spacedmonkey, @stevenkword

Chat Summary:

  • #41936: It was decided that the new filter pre_get_main_site_id should not actually set the WP_Network::$blog_id property when used and only override the return value when accessing it, as it could otherwise have unexpected consequences if a hook set for it was (temporarily) removed. The latest patch had one issue where a variable had not been correctly renamed after moving the code from the function to the class method. It was furthermore discussed how verbose and strict casting the value returned by a filter should be. It was decided to only return the filter value if 0 < (int) $value. Both issues have since been fixed in the latest patch.
  • #38570 and #41652: @sergey is taking care of these i18n tickets.
  • #41789: @johnbillion will provide feedback.
  • As there is now a 5.0 milestone on Trac, the priority tickets that were previously flagged with “Future Release” and “early” are now being moved into the actual milestone.
  • #40364: @flixos90 asked for feedback for this rather complex ticket, as it should preferably get ready early in the 5.0 cycle. That ticket should be discussed in detail in one of the next few weeks, once the multisite work for 4.9 has wrapped up.

Next meeting

The next office hours will take place on October 17th, 2017, 16:00 UTC. Its agenda will be to continue discussing 4.9 tickets and the dev-note for the Networks & Sites component.

Ticket Scrub Recap

The agenda for this ticket scrub was to review 4.9 bug and task tickets that still need to be finished and merged within Beta, similar to the office hours meeting agenda this week (see above).

The meeting’s chat log

Attendees: @afercia, @flixos90, @jeremyfelt, @jjj, @paaljoachim, @spacedmonkey

Chat Summary:

  • #41936: It was decided that moving the logic that is currently in get_main_site_id() is okay to be moved to WP_Network:get_main_site_id(), since that data is very specific to each individual network. It must furthermore be ensured that all values are properly typecast into the expected type (WP_Network::$blog_id is a string, WP_Network::$site_id is an integer and WP_Network::get_main_site_id() should always return an integer). Minor tweaks suggested were returning get_current_blog_id() instead of a hardcoded value of 1 in get_main_site_id() for non-multisite environments and using an internal variable for the cache key used. @spacedmonkey and @flixos90 will make sure this gets ready.
  • #42093: @jeremyfelt has been working on providing an easy way to run unit tests for a subdomain install. This ticket is currently a task scheduled for 4.9, but may as well be punted to 5.0 in case it does not get ready in time.
  • #39419: It was agreed on that the doc block for the globals should indicate replacements to use for those that are deprecated. @jeremyfelt has since updated the patch. It is still being considered whether globals such as $table_prefix, which are used there, but not actually set up there initially should be listed as well.
  • #41789: It was briefly discussed how to be most precise about the documentation of the get_sites() (and related WP_Site_Query method) return value, without making the description overly complex. @jeremyfelt added the final idea as a comment on the ticket and is now waiting for feedback. It was also mentioned that documentation for the other query classes should probably be changed in a similar way, however the ticket for now should only deal with sites and networks.

Next meeting

The next ticket scrub will take place on October 16th, 2017, 17:00 UTC. Its agenda will be to continue discussing 4.9 tickets, particularly the issue that came up with #40228.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #networks-sites, #summary

Multisite Recap for the week of September 25th

Office Hours Recap

No office hours meeting took place this week.

Next meeting

The next office hours will take place on October 3rd, 2017, 16:00 UTC. Its agenda will be to get the 4.9 enhancements ready to be merged before Beta, and then coordinate remaining work on 4.9 bug tickets.

Ticket Scrub Recap

The agenda for this ticket scrub was to look through the multisite tickets without a response.

The meeting’s chat log

Attendees: @afragen, @flixos90, @schlessera, @spacedmonkey

Chat Summary:

  • The issue with the suggestion in #41443 is that wpmu_validate_blog_signup() is supposed to validate new sites submitted by regular users, not by administrators, so that function would not be applicable for usage on the New Site network admin page. However, some restrictions for new user-submitted sites do not apply to admin-created sites, so these restrictions should be reviewed and possibly adjusted to work more similarly to each other.
  • #41685 is a very early ticket, since it requires the site meta API and related blogmeta database table in order to be worked on. It was agreed on that it would generally make sense to store site database versions in the new table instead of an entirely separate table. However, since these versions are not used anywhere in core and since it is questionable whether the blog_versions table can be removed for backward-compatibility reasons, this still needs a deeper discussion once site meta is actually part of core.
  • #41936 was figured out to be a minor bug with the new get_main_site_id() not taking the possibly already set WP_Network::$blog_id property into account. This should definitely be fixed for 4.9. The current patch by @spacedmonkey moves the logic from the function to the WP_Network class method, which is not preferable, but might end up being the only way to accomplish the goal without running into an infinite loop issue. @flixos90 will try to find a better way to fix the issue, but if nobody finds one, the current patch is solid enough to go in. This can happen last minute though, as this will be a bug-fix and does not need to be merged before beta.

Next meeting

The next ticket scrub will take place on October 2nd, 2017, 17:00 UTC. Its agenda will be reviewing the 4.9 enhancements and coordinating who works on the possibly required changes so that another review can happen during the office hours a day later.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #networks-sites, #summary

Multisite Recap for the week of September 11th

Office Hours Recap

The agenda for this office hours meeting was to resolve the remaining discussion and blockers for #29684, the proposed get_main_site_id() function and related integration.

The meeting’s chat log

Attendees: @afragen, @desrosj, @flixos90, @jeremyfelt, @johnjamesjacoby, @spacedmonkey

Chat Summary:

  • The main purpose this function could serve initially is to auto-fill the $blog_id property of the WP_Network class, which currently is only being populated for the current network in the bootstrap process. There is no database field for the main site of a network, therefore custom logic must run for it to be set. get_main_site_id() makes it easy to detect the main site for a network, and magic getters in WP_Network would allow to automatically set the property once it is first accessed, using the new function. In the end of the discussion it was decided that the logic to auto-fill the property makes sense and can be merged with the new function.
  • It was also discussed whether a network option should be used to store the main site ID. This would bring a performance benefit for multisite setups without an object cache and without the network constants enabled. On the other hand, the main site ID at this point is not really an option, since many areas of WordPress assume it is always the site whose domain and path match the network’s ones. Getting rid of this restriction is something that could be evaluated more closely in the future, but for now this restriction exists, and introducing a network option would give the impression that it would be possible to change the main site ID without any issues. Therefore it was decided to not use a network option at this point, but it can be reconsidered later in a separate ticket.
  • Another topic was whether the actual logic should go into the get_main_site_id() function or whether the function is not required at all and instead the logic should be part of WP_Network. Eventually it was agreed to go with the regular function and call it from WP_Network. Moving the logic into a private WP_Network method would not align well with existing core patterns, where pretty much everything relies on a function. As long as there is no methodological approach for this, functions should remain the source as it currently is.
  • Naming of the new function was discussed as well. It was suggested to call the function get_main_site_id_for_network() or get_network_main_site_id() to be more precise. On the other hand, is_main_site() already exists and would have been called similarly in order to align with the new function. Furthermore the function is available for non-multisites, making the extra network affix a bit more confusing. It was decided to proceed with the current idea of get_main_site_id().
  • All remaining items were solved and as of now the patch has been merged into core.

Next meeting

The next office hours will take place on September 19, 2017, 16:00 UTC. Its agenda will be to further plan 4.9 work and which tickets should receive the main focus in the few remaining weeks until Beta 1.

Ticket Scrub Recap

The agenda for this ticket scrub was to review and discuss some multisite tickets in the 4.9 milestone.

The meeting’s chat log

Attendees: @afragen, @desrosj, @flixos90, @jeremyfelt, @sergey, @vizkr

Chat Summary:

  • The first ticket discussed was #40764: @afragen asked for feedback. @flixos90 verified that the latest patch applies cleanly. @desrosj plans to review the patch itself soon.
  • #41285 was discussed: @jeremyfelt was asked for feedback and responded that he is confident that this change can happen, at least for the $public global. He will dive deeper into whether the $site_id global is safe to remove as well. He furthermore stated that the related tickets #34217 and #39419 should be considered as well. A response from Automatticians working on wordpress.com would be much appreciated.
  • #40364 was the last ticket for the meeting: The proposed action hook names used in the new functions wp_insert_site(), wp_update_site() and wp_delete_site() using the same names were questioned and whether it may be more useful to use more precise names using past tense, such as wp_inserted_site(), so that it is clear they run after the database transaction. It was also considered to run multiple actions. Eventually it was decided to go with the simple approach for now, and stick with one action having the function name. Regarding timing, while the latest patch may possibly be merged at this point, it should rather wait until #41333 has also been completed, to have the full new site API in core in one release. The latter ticket is something that can be worked on during the 4.9 beta, to aim for an early 5.0 merge.

Next meeting

The next ticket scrub will take place on September 18, 2017, 17:00 UTC. Its agenda will be to review multisite bug tickets awaiting review with a focus on recently opened tickets.

If you were unable to attend one of these meetings but have feedback, please share your thoughts in the comments on this post. In case there’s a need for further discussion we will ensure to make time for it in one of next week’s chats. See you next week!

#4-9, #multisite, #networks-sites, #summary

Multisite Agenda for the week of September 11th

Office Hours Agenda

This is the agenda for the weekly office hours meeting on September 12th, 2017, 16:00 UTC in #core-multisite.

  • Further discuss the approach on #29684, particularly the following questions:
    • Is get_main_site_id() useful enough on its own to merge it into core as a first step?
    • Is storing the main site ID in a network option worth it? How much do the network meta efforts (particularly #37181) affect this decision?
  • Plan further 4.9 organization as Beta 1 is getting closer.

Ticket Scrub Agenda

This is the agenda for the weekly ticket scrub meeting on September 11th, 2017, 17:00 UTC in #core-multisite.

  • Discuss and review one or two of the 4.9 tickets in more detail, depending on the interest of the meeting’s attendees.

Please join the chat if you’re interested in one of the topics. In case you cannot make the respective meeting, we will be working on publishing a recap post afterwards. If you have some thoughts beforehand or would like something related to be part of the agenda, feel free to share your ideas in the comments for this post. See you in the chat!

#4-9, #agenda, #multsite, #networks-sites

Multisite Agenda for the week of August 28th

Office Hours Agenda

This is the agenda for the weekly office hours meeting on Tuesday 16:00 UTC in #core-multisite.

  • Review progress on roadmap and related discussion.
  • Figure out how to check for existence of the wp_blogs database table for site meta (see #37923). This was discussed last week, but there is still some arguing in the ticket.
  • Open floor if there is time left.

Ticket Scrub Agenda

This is the agenda for the weekly ticket scrub meeting on Monday 17:00 UTC in #core-multisite.

  • Review progress on 4.9 tickets.
  • Open floor for any multisite-related bugs and small enhancements.

Please join the chat if you’re interested in one of the topics. In case you cannot make the respective meeting, we will be working on publishing a recap post afterwards. If you have some thoughts beforehand or would like something related to be part of the agenda, feel free to share your ideas in the comments for this post. See you in the chat!

#4-9, #agenda, #multisite, #networks-sites