XML Sitemaps Meeting: May 26th, 2020

A lot has been going on since our last blog post. Besides many improvements and fruitful discussions in the XML Sitemaps feature project, the proposed schedule for WordPress 5.5 has been published.

The first betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. is currently slated for July 7, which still gives us a bit of time to finish work on our project. We’re getting close to a state where we feel comfortable publishing a merge proposal for inclusion in WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. as part of the 5.5 releaseRelease A release is the distribution of the final version of an application. A software release may be either public or private and generally constitutes the initial or new generation of a new or upgraded application. A release is preceded by the distribution of alpha and then beta versions of the software..

This post aims to give an overview of the things currently in progress, and the items that should be discussed in the upcoming meeting on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Updates

  • Custom elements in sitemap (#151)
    The pull request proposing a simple APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. still needs some documentation, but overall ready for review.
  • PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 Warning on PHP < 7.3 (#186)
    A pull request has been opened to fix this edge case. Ready for review.
  • Filters for individual sitemap entries (#191)
    Freshly proposed over the weekend, this PR makes it easier to modify sitemap data while having access to the necessary post/term/user objects for context. Ready for review.
  • FilterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. to modify WP_Query arguments (#183)
    Feedback from initial code review still needs to be incorporated.
  • Removing core_ prefixes from code (#182)
    Still some feedback that needs to be incorporated. Tests are currently not passing. Otherwise very close.
  • Custom elements in stylesheet (#152)
    PR #163 is still work in progress. The feeling last time was that it might be too complex for core, and perhaps better off as a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party.
    If the pre-existing stylesheet is deemed to not satisfy the needs, we could also omit the stylesheet completely if wanted.
  • Merge proposal post
    We started with an early draft for this, and will share the draft post with the broader group once we feel more comfortable with it.
  • Core patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing.
    Not yet in progress, as there are some bigger outstanding PRs needed for this. We’ll likely start with a first proof-of-concept PR that can then serve as the basis for discussion.

Agenda: May 26th

The next meeting will be held on Tuesday, May 26th at 16.00 CEST.

Items on the agenda so far:

  • Going through items from updates above
  • Open floor

Want to add anything to the above? Please leave a comment here or reach out on Slack.

This meeting is held in the #core-sitemaps channel. To join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: May 12th, 2020

It’s been a while since the last blog post about the XML Sitemaps feature project. Today I’d like to summarize the developments since last month, and also post the agenda for our upcoming meeting.

Updates

  • Version 3.0 (#181)
    A pull request has been opened to add the change log for our next releaseRelease A release is the distribution of the final version of an application. A software release may be either public or private and generally constitutes the initial or new generation of a new or upgraded application. A release is preceded by the distribution of alpha and then beta versions of the software., which includes a good amount of improvements over the last version. The goal is for this to be one of the last releases before we put our efforts into a merge proposal.
    Most notable changes: important refactorings, rewrite rule fixes, removal of lastmod.
  • Custom elements in sitemap (#151)
    There is no consensus yet about what the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. for adding custom elements (beyond lastmod or priority) could look like, especially when dealing with custom namespaces.
  • Custom elements in stylesheet (#152)
    There is still an open PR that tries to makemake A collection of P2 blogs at make.wordpress.org, which are the home to a number of contributor groups, including core development (make/core, formerly "wpdevel"), the UI working group (make/ui), translators (make/polyglots), the theme reviewers (make/themes), resources for plugin authors (make/plugins), and the accessibility working group (make/accessibility). the XSLT stylesheet more dynamic so it automatically displays sitemap elements added by plugins. That means there’s some overlap with the issue mentioned above.
    Interesting side affect: the PR proposes a new esc_xml() function that might be worth looking into.
  • Road towards WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. (#164)
    We’re full steam ahead on improving the source code so it better aligns with WordPress core standards, as well as getting feedback from various core teams on the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party (i.e. polyglots, accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility), docs). Also, there is now a TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. ticketticket Created for both bug reports and feature development on the bug tracker. to reference! See #50117.

Agenda: May 12th

The next meeting will be held on Tuesday, May 12th at 16.00 CEST.

Items on the agenda so far:

  • Version 3.0 (#181)
  • Custom elements
  • Stylesheets
  • Currently open issues and pull requests. Including:
    • Removing trailing slashes on URLs (#134)
    • Sanitizing object subtypes for use in URLs (#166)
    • Adding filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. for WP_Query args (#131) (needs owner)
  • Current roadmap status
  • Open floor

Want to add anything to the above? Please leave a comment here or reach out on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

This meeting is held in the #core-sitemaps channel. To join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: April 14th, 2020

Meeting Recap: April 7th

For reference, check out the previous blogblog (versus network, site) post from April 7th:

What we’ve discussed last week:

  • PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party Conflicts (#146)
    We reached the conclusion that such conflicts are actually a non-issue. Plugins are expected to override the default sitemap functionality. For consistency reasons, we keep the wp- prefix.
  • Last modified date (#145)
    There is one open question on the PR to keep lastmod for object types that support it out of the box (i.e. posts).
    Current status: needs reviews.
  • Rewrite Rules (#147)
    A change was proposed to improve the way rewrite rules are registered for sitemaps. This would avoid having to flush rewrite rules when custom providers are added.
    Current status: needs contributors / reviews.
  • Roadmap
    WordPress 5.5 is ought to be released in August. We settled on the following roadmap for sitemaps:
    • Fix remaining big issues – April
    • Refactor code to be closer to WP coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. standards, add safeguards so nothing breaks after merge – April
    • Publish Merge proposal – May
  • Extensibility
    It was suggested to add a filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. for the <urlset> element’s attributes so that plugins could easily add namespaced elements to a sitemap (e.g. for image sitemaps).

Agenda: April 14th

The next meeting will be held on Tuesday, April 14th at 16.00 CET.

Items on the agenda so far:

Want to add anything to the above? Please leave a comment here or reach out on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

This meeting is held in the #core-sitemaps channel. To join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: April 7th, 2020

Meeting Recap: March 24th & 31st

For reference, check out the previous blogblog (versus network, site) post from March 24th:

Things that have been discussed in the last two meetings:

  • PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party Conflicts (#146)
    We discussed the rewrite rules conflicts that might currently happen with certain plugins. A consensus hasn’t been reached so far, but it could actually be declared a non-issue: if plugin A overrides the default sitemap functionality, that might be intended behavior.
    Current status: needs decision.
  • Rewrite Rules (#147)
    A change was proposed to improve the way rewrite rules are registered for sitemaps. This would avoid having to flush rewrite rules when custom providers are added.
    Current status: needs contributors.
  • Last modified date (#145)
    The PR to remove all traces of lastmod has been updated to include documentation.
    Current status: needs reviews.
  • Filterable query args
    #137 has been closed in anticipation of an improved solution to filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. query arguments.
    Current status: needs PR.
  • Private sites (#138)
    Current status: PR needs some work
  • Roadmap
    WordPress 5.5 is ought to be released in August. We need to continue working on the feature pluginFeature Plugin A plugin that was created with the intention of eventually being proposed for inclusion in WordPress Core. See Features as Plugins. so we have something merge-able in May or June. The tentative roadmap would be:
    • Fix remaining big issues – April
    • Refactor code to be closer to WP coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. standards, add safeguards so nothing breaks after merge – April
    • Publish Merge proposal – May

Agenda: April 7th

The next meeting will be held on Tuesday, April 6th at 16.00 CET.

Today’s agenda is rather straightforward so far:

  • Plugin conflicts
  • Last modified date (#145)
  • Road towards WordPress 5.5
  • Currently open issues and pull requests
  • Open floor

Want to add anything to the above? Please leave a comment here or reach out on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

This meeting is held in the #core-sitemaps channel , to join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: March 24th, 2020

In case you were looking for an blogblog (versus network, site) post about the XML Sitemaps feature project last week, worry no more. Work on the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party is progressing smoothly and steadily, we just didn’t publish an agenda post last week. That means it is time for a double update today!

Meeting Recap: March 10th & 17th

For reference, check my previous blog post from March 10th:

A lot has happened since then. Here’s the summary, not necessarily in the right order:

  • SimpleXML dependency
    We received great feedback from a variety of big hosting providers, all saying that this PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 extension is widely available and we can rely on it safely.
    Current status: no action needed.
  • Rewrite rule conflictconflict A conflict occurs when a patch changes code that was modified after the patch was created. These patches are considered stale, and will require a refresh of the changes before it can be applied, or the conflicts will need to be resolved. with plugins
    As we realized that the new /wp-sitemap.xml URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org format clashes with big existing plugins, we decided to look into alternate names for both the rewrite rules as well as the query params. See GitHub issue for details.
    Current status: needs contributors.
  • Rewrite rule issues with custom providers
    It was reported that adding custom sitemap providers might require flushing rewrite rules. Ideally, that shouldn’t be needed.
    Current status: needs decision.
  • Last modified date (lastmod)
    We decided to continue with the proposed PR to remove lastmod from sitemaps (at least for now), but need to makemake A collection of P2 blogs at make.wordpress.org, which are the home to a number of contributor groups, including core development (make/core, formerly "wpdevel"), the UI working group (make/ui), translators (make/polyglots), the theme reviewers (make/themes), resources for plugin authors (make/plugins), and the accessibility working group (make/accessibility). sure there is appropriate documentation. It’s something that can always be added back if needed.
    Current status: has PR, needs documentation.
  • Query Filters
    Valuable feedback emerged from testing, which led to the decision to close the existing PR to make query instances filterable in favor of a simpler approach. In its place, we should make the query arguments filterable, and also add filters to short-circuit queries.
    Current status: needs contributors.

Please let me know in the comments if I got something wrong in this summary!

Agenda: March 24th

The next meeting will be held on Tuesday, March 24 at 16.00 CET.

Today’s agenda is rather straightforward so far:

Want to add anything to the above? Please leave a comment here or reach out on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

This meeting is held in the #core-sitemaps channel , to join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: March 10th, 2020

A lot has happened since last week’s meeting for the XML Sitemaps feature project. Here’s a quick rundown of what we’ve discussed & did, as well as a brief agenda for today’s meeting.

Meeting Recap: March 3rd

For reference, please check out last week’s agenda post:

The tl;dr of our discussion:

  • Disabling sitemaps for private sites
    Mentioned the currently open PR and how it could be used to kill two birds with one stone by making that process filterable; thus making it easier for plugins to disable the sitemaps feature.
    Current status: needs tests
  • Prefixing sitemap URLs
    The main PR for this change has been merged, a new issue has been opened for @kraftbj to handle 404 requests.
  • SimpleXML dependency
    We went over potential alternatives to this extension, but ultimately settled on sticking with the status quo as initial feedback indicated a rather wide availability of SimpleXML. We then discussed how we should gracefully handle the unavailability of said extension and decided on using wp_die to output a nicely formatted error message in XML with HTTPHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. status 501 (“Not implemented”).
    Current status: merged!
  • @joemcgill proposed looking into how to best transition the code base to something more in line with WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.. Something that we can discuss in a future meeting, once the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party is more stable.
  • Added @pbiron, @kraftbj, and @pfefferle as new contributors to the GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ repository. 🎉

Agenda: March 10th

The next meeting will be held on Tuesday, March 10 at 16.00 CET.

PSA: Unfortunately I won’t be able to lead today’s meeting, but thankfully @tweetythierry stepped up to help out with this.

Today’s agenda is rather straightforward so far:

  • Released version 0.2.0 of the plugin (changelog)
  • Plugin compatibility with new URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org structure
    Yoast SEO’s rewrite rules seem to clash with ours
  • SimpleXML dependency: blogblog (versus network, site) post on makemake A collection of P2 blogs at make.wordpress.org, which are the home to a number of contributor groups, including core development (make/core, formerly "wpdevel"), the UI working group (make/ui), translators (make/polyglots), the theme reviewers (make/themes), resources for plugin authors (make/plugins), and the accessibility working group (make/accessibility)./hosting (@pbiron)
  • Currently open issues and pull requests
  • Open floor

Want to add anything to the above? Please leave a comment here or reach out on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

This meeting is held in the #core-sitemaps channel , to join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: March 3rd, 2020

Another week passed by with quite a productive meeting for the XML Sitemaps feature project. Here’s a short summary, as well as the agenda for today’s meeting.

Meeting Recap: February 25th

In case you missed it, I recommend checking out last week’s post with everything that happened so far:

As planned, we went over some of the existing issues, but we also discussed some things that came up on short notice. Here’s the gist:

  • We reiterated on the idea to remove the lastmod field. @swissspidy offered to start a PR that explores this so it can actually be tested in the wild. @joemcgill offered to post some stats about the performance of this last modified date calculation.
  • There was a discussion, also after the meeting, about changing URLs to have a /wp- prefix and whether that prefix should be filterable. The consensus was that a filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. is unnecessary. A new PR was added to implement this.
    @kraftbj offered his help to implement automatic redirects from /sitemap.xml to /wp-sitemap.xml for improved discoverability.
  • Next up was the SimpleXML dependency and how the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party should behave when that PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 extension is missing.
    We tend towards just disabling sitemaps if that’s the case, but perhaps provide some messaging about it.
    @kraftbj offered to try to get some stats about the availability of SimpleXML via Jetpack, as well as to help with a PR.
    @pbiron reached out on the hosting community channel, and is looking for specific questions that we could ask in a make/hosting post.
  • Last but not least, there was an open question about leveraging the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. for sitemaps. It was not fully clear though how that would be beneficial. As of now, there are no plans to explore this.

Agenda: March 3rd

The next meeting will be held on Tuesday, March 3 at 16.00 CET

This meeting is held in the #core-sitemaps channel , to join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #xml-sitemaps

XML Sitemaps Meeting: February 25th, 2020

Last week we held the first of many weekly meetings for the XML Sitemaps feature project on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Meeting Recap: February 18th

We had quite a few people attending, not all of whom were familiar with the project. Thus, we started off with a small recap of the project’s scope and goals. After that we discussed various different topics:

  • How to modify the sitemaps to include/exclude certain URLS
    A pull request has been opened to add a FAQ section to the readme that aims to answer these kind of questions.
    Also, a new way to filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. WP_Query instances used for sitemaps has been proposed.
  • Why are there no changefreq and priority fields?
    Those are optional fields in the sitemaps protocol and not typically consumed by search engines. The feature pluginFeature Plugin A plugin that was created with the intention of eventually being proposed for inclusion in WordPress Core. See Features as Plugins. follows other solutions like Yoast SEO who also don’t include those fields.
    Developers can still add those fields if they really want too.
  • Will there be UIUI User interface controls to include/exclude content from sitemaps?
    Adding UI controls is currently a non-goal for the project.
  • Calculating the last modified date for URLs
    This is rather difficult and computationally expensive in WordPress. Given that sitemaps are first and foremost a discovery mechanism for content, having this data is not necessarily required. We will explore omitting this functionality (GitHub issue).
  • The default limit of 2000 URLs per sitemap is considered high and might need to be re-evaluated.
  • Potential compatibility issues with other XML Sitemaps plugins have been discussed.
    If a sitesite (versus network, blog) ends up having two sitemaps by accident that wouldn’t be bad. However, the current /sitemap.xml URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org might clash with other plugins. A GitHub issue has been opened to suggesting using /wp-sitemap.xml as the base. This would avoid conflicts in this regard.

Agenda: February 25th

The next meeting will be held on Tuesday, February 25 at 16.00 CET

For tomorrow’s meeting, the agenda is rather brief:

  • Updates since last week (merged changes, new issues)
  • Next steps for proposed lastmod changes
  • Next steps for URL naming change
  • Planning releaseRelease A release is the distribution of the final version of an application. A software release may be either public or private and generally constitutes the initial or new generation of a new or upgraded application. A release is preceded by the distribution of alpha and then beta versions of the software. of version 0.2.0

This meeting is held in the #core-sitemaps channel , to join the meeting, you’ll need an account on the Making WordPress Slack.

#agenda, #feature-plugins, #feature-projects, #seo, #xml-sitemaps

XML Sitemaps Kickoff Meeting Announcement

A few weeks ago an update was posted for the XML Sitemaps feature project to give everyone an idea of where it is heading.

Now, we want to gather more contributors around the feature pluginFeature Plugin A plugin that was created with the intention of eventually being proposed for inclusion in WordPress Core. See Features as Plugins. and get your feedback on the project. For this, we’re kicking off regular meetings in the brand new #core-sitemaps SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel.

The first meeting will be held on Tuesday, February 18 at 16.00 CET and will serve as an introduction to the project and an opportunity to discuss the next steps. As such, there is currently no formal agenda for this inaugural meeting.

However, if you have anything specific that you’d like to propose being discussed in this meeting, feel free to leave a comment below.

This meeting is held in the #core-sitemaps channel , to join the meeting, you’ll need an account on the Making WordPress Slack.

#feature-plugins, #feature-projects, #seo, #xml-sitemaps

Feature Plugin: XML Sitemaps

Last year, a group of contributors posted a proposal to implement native XML Sitemaps in WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. which received lots of interest and feedback from the community. Since then, we have been working on a XML Sitemap feature plugin (MVPMinimum Viable Product "A minimum viable product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development." - WikiPedia) which is now available for testing and feedback.

Props to the contributors working on this pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and co-authoring the content of this post: Sander van Dragt, Kirsty Burgoine, Adrian McShane, Ruxandra Gradina, Joe McGill, Thierry Muller, Pascal Birchler 

Feature overview

As a quick reminder of what this project is trying to achieve, here are the main features as described in the initial project proposal, which we would encourage you to read in its entirety.

XML Sitemaps will be enabled by default making the following content types indexable

– Homepage
– Posts page
– Core Post Types (Pages and Posts)
– Custom Post Types
– Core Taxonomies (Tags and Categories)
– Custom Taxonomies
– Users (Authors)

Additionally, the robots.txt file exposed by WordPress will reference the sitemap index.

Additionally, an XML Sitemaps APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. ships with the plugin aiming for developers to build on top of it. 

The approach

In order to fulfil these initial requirements, we researched the way several existing popular plugins implement this functionality, and came up with an approach that we believe combines many of the best ideas from each.

The sitemap index

A crucial feature of the sitemap plugin is the sitemap index. This is the main XML file that contains the listing of all the sitemap pages exposed by your WordPress sitesite (versus network, blog) and the time each was last modified. By default, the plugin creates a sitemap index at /sitemap.xml which includes sitemaps for all supported content, separated into groups by post types, taxonomies, and users.

Sitemap pages

Each sitemap page will be available at a URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org using the following structure, sitemap-{object-type}-{object-subtype}-{page}.xml. Some examples of this structure applied to real content include:

  • Post type – posts: sitemap-posts-post-1.xml 
  • Post type – pages: sitemap-posts-page-1.xml
  • TaxonomyTaxonomy A taxonomy is a way to group things together. In WordPress, some common taxonomies are category, link, tag, or post format. https://codex.wordpress.org/Taxonomies#Default_Taxonomies. – categories: sitemap-taxonomies-category-1.xml
  • Users – sitemap-users-1.xml (note that the WP_User object doesn’t support sub-types)

The official sitemaps protocol asserts that each sitemap can contain a maximum of 50,000 URLs and must be no larger than 50MB (52,428,800 bytes). However, in practice, we found that performance begins to degrade when trying to generate a query that returns more than a few thousand URLs, so for that reason, we’ve decided to limit the default implementation to a maximum of 2,000 URLs per sitemap, which can be modified by using a filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. on the core_sitemaps_max_urls hook.

Sitemap pages for each public post type (except attachments) will be generated, which include URLs to individual post pages. Likewise, sitemaps will be generated for all public taxonomies, which include URLs to taxonomy archive pages, and sitemaps will be generated for all users with published public posts, which includes the URL for each user’s author archive page. The list of supported sub-types for posts and taxonomies can be filtered using the core_sitemaps_post_types and core_sitemaps_taxonomies filters, respectively. Additionally, URLs for any object type can be added or removed using the following filters:

  • Post types: core_sitemaps_posts_url_list
  • Taxonomies: core_sitemaps_taxonomies_url_list
  • Users: core_sitemaps_users_url_list

Performance and scalability

Adding an XML Sitemaps caching mechanism was specifically listed as a non-goal of the project, so we have not included one. However, we did want to ensure that the initial version of the plugin took scalability into consideration, so we spent time researching the major scalability issues present in current popular implementations and ways of solving those problems.

By using best practices for making our main queries performant, we were able to eliminate most scalability problems from individual sitemap pages. However, the main performance problem is generating last modified times for each page in the sitemap index. It’s not scalable to calculate these values dynamically, so instead, we’ve started with an implementation that updates these values using a WP_Cron task that runs twice daily, and saves these values in the options table.

We’ve also begun researching and writing up an implementation for a more robust sitemap page caching mechanism, using custom post types to store and update sitemap data, which can be further explored if the initial implementation proves to be insufficient as an initial implementation for core. (See: #1 and #39 for more details).

Next steps

Announcing the first version of this feature plugin is a major milestone, but is only an early step in the process of having this functionality included in WordPress Core. Now is when we need your help to test, validate, and improve what we have built to ensure that we meet the needs of the broad WordPress community. We are also encouraging sitemaps plugins authors to integrate with the plugin, specifically leveraging the sitemaps API to extend its core functionalities.

We will kick start weekly meetings on WordPress SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. in the very near future. In the meantime, we would encourage anyone interested to join now and begin discussion about this feature. Additionally, you can leave questions and feedback in the comments section of this post or as new issues on the GitHub repo.

Thanks for reading!

#feature-plugins, #feature-projects, #seo, #xml-sitemaps