WordPress 5.7 introduces a filter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.-based Robots API An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways., providing central control over the robots meta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. tag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.). It furthermore adds a max-image-preview:large directive to the tag by default.
The robots meta tag lets you utilize a granular, page-specific approach to controlling how an individual page should be indexed and served to users in search engine results. This meta tag is commonly placed in the <head> section of a given page:
<!DOCTYPE html>
<html>
<head>
<meta name="robots" content="max-image-preview:large" />
</head>
The max-image-preview:large directive allows search engines to display large image previews for the site resulting in an enhanced user experience. The directive is now added by default to sites that wish to be indexed by search engines.
wp_robots() new function
WP 5.7 introduces a wp_robots() function which outputs a robots meta tag. The function is automatically integrated with the relevant areas in WordPress core Core is the set of software required to run WordPress. The Core Development Team builds WordPress., for example it is added to wp_head so that the robots meta tag is available in the frontend. Plugins and themes should not typically call this function manually; for special cases though, e.g. a custom frontend template that does not use wp_head(), the new function can be hooked into a custom filter:
add_action( 'my_custom_template_head', 'wp_robots' );
How to hook into the Robots API
The robots meta tag attributes can be filtered using the wp_robots hook. The wp_robots() function is entirely filter-based, so if no filter is added to wp_robots, no directives will be present, and therefore the entire robots meta tag will be omitted.
Plugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party developers that add their own robots meta tags are encouraged to switch to using the new Robots API, hooking their custom directives into the wp_robots filter.
Default robots meta tag values
As mentioned above, the robots meta tag is now included by default in wp_head as follows for WordPress sites that wish to be indexed by search engines:
<meta name="robots" content="max-image-preview:large" />
Developers can disable this directive using the following PHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher snippet:
remove_filter( 'wp_robots', 'wp_robots_max_image_preview_large' );
The Robots API is also used to generate robots meta tags on some specific screens, with the following values:
- When the “Discourage search engines from indexing this site” setting is enabled, the front end includes a
noindex,nofollow directive in the Robots meta tag, as in previous versions. In this case, the max-image-preview:large directive is not added. - Within the Customizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings. preview, the site includes a
noindex directive in the Robots meta tag, as in previous versions. - The WordPress login page (
wp-activate.php) includes a noindex,noarchive directive in the Robots meta tag, as well as a <meta name="referrer" content="strict-origin-when-cross-origin" /> tag, as in previous versions. - In multisite Used to describe a WordPress installation with a network of multiple blogs, grouped by sites. This installation type has shared users tables, and creates separate database tables for each blog (wp_posts becomes wp_0_posts). See also network, blog, site, the site activation page (
wp-activate.php), where a newly registered user can confirm their newly created site, includes a noindex,noarchive directive in the Robots meta tag, as well as a <meta name="referrer" content="strict-origin-when-cross-origin" /> tag, as in previous versions.
Add custom values to robots meta tag
The wp_robots filter takes an array of key-value pairs. To add custom values to the robots meta tag, use the following filter:
function wporg_wp_robots_add_follow( $robots ) {
$robots['follow'] = true;
return $robots;
}
add_filter( 'wp_robots', 'wporg_wp_robots_add_follow' );
It will add the follow value in addition to any existing values. Given the max-image-preview:large is already added by default, this snippet will generate the following:
<meta name="robots" content="max-image-preview:large, follow" />
It’s also possible to unset existing values:
function wporg_wp_robots_add_follow( $robots ) {
unset( $robots['max-image-preview'] );
$robots['follow'] = true;
return $robots;
}
add_filter( 'wp_robots', 'wporg_wp_robots_add_follow' );
Deprecated functions
The new Robots API controlled via the wp_robots filter replaces various separately injected robots meta tags that WordPress core used to conditionally provide. As such, the following functions have been deprecated and should no longer be used:
noindex(); instead, the new wp_robots_noindex() function can be hooked into the wp_robots filter.wp_no_robots(); instead, the new wp_robots_no_robots() function can be hooked into the wp_robots filter.wp_sensitive_page_meta(); this function placed two tags, so it has two separate replacements:- For the
noindex,noarchive robots directive, the new wp_robots_sensitive_page() function can be hooked into the wp_robots filter. - For the extra
referrer tag, the new wp_strict_cross_origin_referrer() function can be called.
For reference, see Trac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. ticket Created for both bug reports and feature development on the bug tracker. #51511
Props to @davidbaumwald, @flixos90, and @tweetythierry for proofreading
#5-7, #dev-notes
