WP_Term_Query in WordPress 4.6

WordPress 4.6 will introduce WP_Term_Query. This new class brings parity between taxonomy term queries and WP’s other content type queries: WP_Query, WP_Comment_Query, and WP_User_Query. And – as in the case of posts, comments, and users – the get_terms() function has been converted to a wrapper for the new WP_Term_Query.

Aside from support for querying by term_taxonomy_id #37074, no functional changes to get_terms() will be introduced in 4.6; 100% backward compatibility with previous uses is the goal for this release.

For more background on the change, see #35381.

#4-6, #dev-notes, #taxonomy

Taxonomy bug scrub summary, 2016-05-05

We had a taxonomy component bug scrub today. Slack archive: https://wordpress.slack.com/archives/core/p1462464023003714

Present: @boonebgorges, @helen, @barryceelen, @wonderboymusic. @swissspidy and @jans-1 voiced their opinions via emoji reactions.

We cleared 8 items from the Awaiting Review queue, bringing it down to 35 tickets.

Let’s do this again real soon!

#4-6, #bug-scrub, #taxonomy

Taxonomy Component Bug Scrub – May 5, 2016

A bug scrub focused on open tickets in the Taxonomy component will be held in the #core channel on Slack at May 5, 2016 16:00 UTC. We’ll focus on tickets languishing in Awaiting Review, and/or any specific tickets that attendees may want to chat about. People from all categories, however you may term yourself, are invited to attend and help us slug things out.

#bug-scrub, #taxonomy

Changes to the Term Edit Page in WordPress 4.5

Up until recently, the term list table and the term edit form in the admin shared the same page: wp-admin/edit-tags.php. This is inconsistent compared to how the post list table and the editor are split up between wp-admin/edit.php and wp-admin/post.php.

As was reported in #34988, this inconsistency led to some problems when screen options belonging to the list table were shown on the term edit page. This was changed in [36308] by introducing wp-admin/term.php 💪🏽.

What this means for developers

First of all, you’ll notice that the links to edit a single term now look like example.com/wp-admin/term.php?tag_ID=123. (previously: example.com/wp-admin/edit-tags.php?action=edit&taxonomy=post_tag&tag_ID=127). That’s not a big deal actually, but it leads to some changes under the hood:

If you’re specifically targeting the term edit form in your plugin, $pagenow changes from edit-tags.php to term.php. The screen base (returned by get_current_screen()) changes from edit-tags to term as well. That’s it 🙂

What to look for

If you want to specifically enqueue scripts and styles on the term edit page, you should hook to load-term.php instead of load-edit-tags.php. That way you won’t unnecessarily load assets on the wrong screen.

Besides that, everything stays the same 😇. If you do however find any quirks related to this change, do not hesitate to leave a comment on this post or the relevant ticket: #34988

#4-5, #dev-notes, #taxonomy

4.4 Taxonomy Roundup

We’ve made lots of improvements to the Taxonomy component for WordPress 4.4. Let’s round ’em up, pardner!🐴

Term Meta

The most significant improvement is the introduction of term meta #10142. Use the new term meta API – add_term_meta(), update_term_meta(), delete_term_meta(), and get_term_meta() – to store arbitrary data about taxonomy terms, in the same way you would for posts, users, or comments. The term query functions get_terms() and wp_get_object_terms() now support a ‘meta_query’ parameter, with syntax identical to the corresponding argument for WP_Query, etc.

Term meta was built with performance in mind. When fetching terms using get_terms() or wp_get_object_terms(), metadata for located terms is loaded into the cache with a single database query; disable this by passing 'update_term_meta_cache' => false in your query parameters. When querying for posts using WP_Query, we have another trick up our sleeves: term meta for terms belonging to matched posts is lazy-loaded, so that all relevant term meta is only fetched the first time you call get_term_meta() in the loop.

Developers who have previously implemented term meta in their own plugins or client sites should prepare their customizations for WordPress 4.4, to ensure that nothing’s broken in the transition.

WP_Term and unique term IDs

For the last few releases, we’ve been working hard on the taxonomy roadmap. A major achievement was the complete splitting of shared taxonomy terms in 4.3, which ensures that terms can be uniquely identified by their term_id. In 4.4, we begin taking advantage of this uniqueness: the $taxonomy parameter is now optional in get_term() and get_term_field(), functions that previously required both $term_id and $taxonomy.

Under the hood, we’ve introduced WP_Term, a new class that properly models a term object #14162. Everywhere where terms can be retrieved – one at a time, as in get_term() or get_term_by(), or in bulk, as in get_terms() and wp_get_object_terms()– you should expect WP_Term objects to be returned, rather than stdClass. For 4.4, this change doesn’t affect much, aside from making our term-caching internals somewhat more sane. But in the future, WP_Term will allow for more extensive caching throughout the Taxonomy component, as well as the potential for method chaining and other developer conveniences.

Other goodies

We’ve made lots of smaller improvements to Taxonomy as well. Some highlights:

  • Registering a taxonomy with 'public => false' now prevents the taxonomy from being queried publicly. (Imagine that!) #21949
  • The $taxonomy parameter for get_term_by() is optional (and ignored) when fetching by term_taxonomy_id. #30620
  • Argument arrays are now filtered in get_terms(), register_taxonomy(), and in the Categories post edit metabox. #33026 #33369
  • Accented characters in term names are no longer ignored when checking for duplicates, allowing for, eg, tags ‘foo’ and ‘fóó’ to coexist. #33864
  • Term relationships caches are properly busted during wp_remove_object_terms(). #34338

#4-4, #dev-notes, #taxonomy

Preparing your plugins and your client sites for termmeta

The biggest hurdle in the introduction of metadata for taxonomy terms (#10142) is compatibility with existing plugins and customizations. In this post, I’ll outline the most significant concerns, along with recommendations for next steps.

This post has two audiences: authors of publicly available plugins and developers of client sites.

I’ve just completed a scan of all the plugins on wordpress.org that mention ‘termmeta’ or ‘term_meta’ (and boy, are my arms tired!). The numbers there are promising – it’s a fairly small number of plugins that will be affected, and many of them have just a few active installations. I’ll share the results of my scan at the end of this post.

Of greater concern are customizations that developers have built for clients. I know that many larger dev agencies, and many individual developers, have custom, in-house libraries that they use for client sites, and many of these libraries implement termmeta in one way or another. The best of these libraries will be unaffected by termmeta in core: they use prefixed functions, they store data in wp_options rather than a custom table, they use function_exists() where appropriate, and so on. However, I’m certain that there are many libraries – in use on many, many WordPress sites – that do not adhere to these best practices. Sites using these libraries will react in unexpected ways – some of which involve fatal errors – if preventative steps are not taken by the developers before termmeta is rolled into core. Developers should update their libraries and client deployments as necessary to avoid lost data or site downtime.

The primary reasons for concern, listed in order of severity, are as follows:

  1. Function name clashes. The proposed core implementation introduces a number of new functions likely to be used in third-party termmeta implementations: add_term_meta(), delete_term_meta(), get_term_meta(), update_term_meta(), and update_termmeta_cache(). (See the latest patches on the Trac ticket for details.) Plugin should either use a prefix (eg myplugin_add_term_meta()), wrap the definitions in function_exists() checks, or disable the termmeta portion of the plugin based on a WP version check.
  2. Termmeta table deletion. A number of plugins have been found that DROP $wpdb->termmeta on plugin deactivation. Aside from being bad general practice, this could be extremely bad in the case that $wpdb->termmeta becomes a core table. Plugins should not drop {$wpdb->prefix}termmeta under any circumstances.
  3. Non-matching function signatures. I haven’t found any specific examples of this, but plugins that define their own versions of unprefixed functions (using function_exists(), as described above, should be sure that the signatures match. For example, if your plugin defines get_term_meta( $term_id, $taxonomy, $key = '', $single = false ), it will break when moving to the core implementation, which has the signature get_term_meta( $term_id, $key = '', $single = false ). Plugin authors should double-check that their use of unprefixed _term_meta() functions matches the parameter order and default values described in the proposed core implementation.
  4. Non-matching table schemas. A number of plugins create tables called {$wpdb->prefix}termmeta. Most custom tables that I’ve seen are very close to the proposed core implementation, aside from a shortened index length on the meta_key column, introduced throughout core in 4.2. In fact, all implementations I’ve seen are close enough that it’s likely that core will be able to continue to use the tables, leaving data intact. However, if there are significant mismatches – for example, different field names – it will cause serious problems. Plugin author should verify that tables called {$wpdb->prefix}termmeta have a compatible schema with the core implementation.

Here’s a list of the plugins I identified from wordpress.org that will be affected by core termmeta (sorted most popular first). All of these plugins violate at least one of the maxims above. A few of them are OK except for a mismatched meta_key index the termmeta table; the current plan is for core to take care of this index automatically on upgrade. Most of the remaining violations have to do with unprefixed function names. If your plugin is listed below, you are strongly urged to put out an update as soon as possible that addresses the concerns described above. (Some of the plugins are very old, and may be inactive and/or unmaintained. In most cases, they should still be updated, in case of legacy sites.) Don’t hesitate to contact me, in the comments or privately, if you need more details about how a specific plugin in the list below will be affected.

It’s looking likely that termmeta will be introduced in WordPress 4.4 (or perhaps 4.5). The time to act is now. Be a good Citizen of the Internet, and fix your plugins sur-le-champ.

#4-4, #roadmaps, #taxonomy

Taxonomy term metadata proposal

During a recent meeting focused on the taxonomy roadmap, a number of contributors discussed the possiblility of adding metadata for taxonomy terms to WordPress #10142. Assuming we decide to build termmeta, a major hurdle will be compatibility with the many plugins – both publicly available and privately developed – that already implement their own version of termmeta. We plan to do an in-depth scan of all public plugins that conflict with the core implementation (function names, database table schema, etc) and to mount an outreach and documentation project to help affected plugin developers move to the core system. Our goal is zero fatal errors or lost data on WP installations.

But before moving forward with research of existing plugins, we need a fairly clear sense of what the core implementation would, in fact, look like. These details will serve as the basis for a set of heuristics developers can use to tell whether their plugins will conflict with what goes into core.

Toward that end, below is the outline of a proposed implementation of termmeta.

  1. Database
    1. Term metadata will be stored in a new database table, called {$wpdb->prefix}termmeta (wp_termmeta, for convenience in this proposal).
    2. The termmeta table name will be stored at $wpdb->termmeta.
    3. The wp_termmeta schema will mirror the schema for postmeta and usermeta tables:
      CREATE TABLE $wpdb->termmeta (
          meta_id bigint(20) unsigned NOT NULL auto_increment,
          term_id bigint(20) unsigned NOT NULL default '0',
          meta_key varchar(255) default NULL,
          meta_value longtext,
          PRIMARY KEY (meta_id),
          KEY term_id (term_id),
          KEY meta_key (meta_key($max_index_length))
      ) $charset_collate;
  2. New API functions
    New API functions will closely mirror those for postmeta and usermeta. They will primarily be wrappers for the _metadata() family of functions. We may also build in some fault tolerance for when $term_id does not identify a unique term (ie, because shared terms have not yet been split by the 4.3 upgrade routine).
    1. add_term_meta( $term_id, $meta_key, $meta_value, $unique = false )
    2. delete_term_meta( $term_id, $meta_key, $meta_value = '' )
    3. get_term_meta( $term_id, $meta_key, $single = false )
    4. update_term_meta( $term_id, $meta_key, $meta_value, $prev_value = '' )
    5. update_termmeta_cache( $term_ids )
  3. Other API improvements
    1. Termmeta cache priming for term-fetching functions: get_term(), get_term_by(), get_terms(), wp_get_object_terms(). See _prime_post_caches() for how this works for postmeta.
    2. A 'update_term_meta_cache' argument for functions that fetch multiple terms (get_terms(), wp_get_object_terms()) that will allow developers to prevent cache priming described in 3.1 above. As in the case of posts, 'update_term_meta_cache' will default to false.
    3. A new 'meta_query' argument for get_terms() and wp_get_object_terms().

Please note that this is a draft. Details are subject to revision based on feedback from contributors. If you see something missing or amiss in the above, please leave a note below. (Please also note that this doesn’t mean that termmeta is a done deal. Sketching the implementation is part of the evaluation process.) Thanks in advance for your feedback!

#proposal, #taxonomy

Taxonomy meeting summary – 2015/09/03

Present: @boonebgorges, @swissspidy, @masonjames, @drewapicture, @georgestephanis, @khromov, @srwells, @michaeltieso, @dpegasusm, @kraft, @mrahmadawais, @samuelsidler, @leatherface_416, @jblz, @tyxla, @jeroenvanwissen, @lindsaymac, @eric, @jbrinley, @brashrebel, @pdufour, @joehoyle, @timothybjacobs, @ryanduff, @krogsgard, @aaroncampbell, @rahe

Logs: https://wordpress.slack.com/archives/core/p1441310435002734

  • Had a general discussion about term meta: who’s used plugins for it, who’s used workarounds, various use cases. We talked a bit about some arguments against term meta: that it will not perform well at scale, that it encourages poor data modeling – but decided that they could be set aside for the most part.
  • Outlined the interpretation, including database table name and schema, function names, and other API additions to support term meta. @boonebgorges will work up a RFC for make/core for feedback.
  • Talked about various ways in which existing term meta libraries might conflict with the core implementation: duplicated function names, duplicate table names, incompatible table schemas, etc. @boonebgorges is assembling a list of plugins in the repo that may conflict with the core implementation. Once the outline of the core implementation is pretty much settled, @aaroncampbell, @krogsgard, @masonjames, and @boonebgorges (and anyone else who is interested) are going to collaborate on reviewing these plugins to see which ones will conflict in serious ways (via a Google Doc, which @boone will share once we’re ready to go). This will help us gauge the extent of potential problems, and get a sense of what outreach will look like.
  • We talked a little about combining the wp_terms and wp_term_taxonomy database tables #30262. We outlined some backward compatibility concerns, and strategies for minimizing conflicts. Put out a general call for thoughts and initial patches on the ticket, though we probably won’t move forward with schema changes for at least one more release cycle.
  • Had a very brief discussion about WP_Term #14162. Initial implementation – probably doable for 4.4 – will be simple, and will focus on strict typing for term data as well as cache invalidation. Future releases may see more functionality moved to the class.

#4-4, #chats, #meeting, #summaries, #taxonomy

Taxonomy roadmap for 4.4 and beyond

In WordPress 4.3, we eliminated shared taxonomy terms once and for all. This means that, by the time WordPress 4.4 is released, just about every WordPress installation will have the same number of rows in the wp_terms and wp_term_taxonomy database tables.

Why does this matter? When terms in different taxonomies could share the same term_id, terms could only be uniquely identified by a ID-taxonomy pair. This is why every nearly function in our taxonomy API has $term_id and $taxonomy as required arguments. (The decision was made long ago to keep the truly-unique term_taxonomy_id internal for most purposes.) The lack of unique IDs for terms made our API interfaces and internals complicated, and it made it cumbersome to add new features like term metadata.

Now that term IDs are unique, we can begin the projects of unraveling the now-unneeded complications of the taxonomy API and adding new features that take advantage of the simplified data model. In this post, I’ll outline some of these tasks, and point to areas where interested folks can contribute.

API simplification and other work on taxonomy internals

Once each row in the wp_terms table corresponds to a single row in wp_term_taxonomy, there’s no point in having two separate tables (and all the JOINs that two tables require). In 2013, @nacin sketched how this might be done, through a combination of $wpdb tricks and a MySQL view. Here, we need someone to write a patch, and we also need to start a discussion about graceful failures for situations where there are still shared terms in the DB, as well as MySQL version compatibility (view functionality was phased in over the 5.0 series). The tracking ticket for simplification of the taxonomy database schema is #30262.

With a single term table, we can begin to rewrite our internal SQL queries to remove costly table joins. This kind of refactoring is probably at least one version (and a hundred unit tests) away. In the meantime, we can begin the process of simplifying the API interfaces. For example, functions that accept term IDs, like get_term() no longer need to require an explicit taxonomy parameter.

Having a unique identifiers for terms means this is also a good time to move toward WP_Term; see #14162. This class can start off being a fairly simple model that takes care of things like basic cache management and data integrity – see WP_Post – but over time, I envision moving business logic to the WP_Term, introducing convenience methods for chaining, and so on.

Term Meta

There’s been lots of clamoring for taxonomy term metadata #10142. Unique term IDs make it viable, since WP’s *_metadata() functions assume object IDs as identifiers.

The technical implementation is not complex. We need wrappers for the CRUD functions, ‘meta_query’ support in get_terms(), an update routine to create the database table, metadata pre-caching (‘update_term_meta_cache’) when fetching terms, and maybe a few other small items.

The larger challenge is to build a core solution that minimizes conflict with third-party tools. Developers have wanted termmeta for a long time, so there are many public plugins and private libraries that provide it. Many of them use unprefixed function names like update_term_meta(), and many of them use a database table called wp_termmeta. We should do a survey of publicly available plugins to get a sense of usage statistics and schema compatibility. We’ll need to organize outreach to developers of known plugins, so that they can add off-switches to their tools before termmeta appears in core. And we may decide to borrow code from one or more of the existing GPL-licensed tools, ideally with the participation of the original author.

Let’s share this journey

Many hands make light work. We need code, but more importantly, we need folks who are nuts about strategy and outreach and backward compatibility.

There’ll be a meeting for all Taxonomy Heroes, on September 3 2015 20:00 UTC, in the #core channel on Slack. If you’re interested in helping out with any of these taxonomy projects, drop a comment below or come to the meeting. We’ll get a team or two together, and make a plan for 4.4 and beyond.

#4-4, #roadmaps, #taxonomy

Eliminating shared taxonomy terms in WordPress 4.3

Work on the taxonomy roadmap, started in earnest during the 4.1 dev cycle, continues to chug along for WordPress 4.3. We’ve been focusing on the elimination of shared terms: terms in different taxonomies that share the same term_id and thus the same row in the wp_terms database table. In 4.1, we stopped creating shared terms. In 4.2, we began splitting existing taxonomy terms when those terms were updated.

In 4.3, we’ll take the final step. When a WordPress site upgrades to 4.3, all existing shared terms will be split.

Toward that end, I’d like to make (a) a final warning to plugin authors, and (b) a plea for help from taxonomy mavens and those who maintain very large and complex WordPress installations.

Plugin authors: Dress rehearsal is nearly over

Since 4.2, saving a shared term (via any interface that uses wp_update_term()) causes that term to be split. It’s fairly rare to update a taxonomy term, which has made 4.2 a sort of trial run for authors to ensure that their plugins are ready for the transition. A quick scan of the plugin repo shows that a few dozen authors have taken advantage of this opportunity. I dub these developers Heroes of WordPress and grant them 🌟🌟 many gold stars 🌟🌟. Many authors, however, have neglected to make the necessary updates.

All WP developers should take note that the upgrade to 4.3 is likely to surface many more odd bugs related to shared terms than 4.2 did. Plugin authors are strongly encouraged to review the 4.2 term splitting field guide to understand the potential repercussions for their public plugins as well as their custom client work, and to review techniques for ensuring a smooth transition.

Get involved

The 4.3 ticket for splitting taxonomy terms on WP upgrade, #30261, is sorely in need of developer feedback. The process of splitting shared terms en masse is potentially resource-intensive, especially on sites with very large numbers of terms (and especially with very large numbers of shared terms). The latest patch on the ticket has some optimizations so that sites with many hundreds of shared terms can be migrated in just a few seconds. If you maintain a site with a large number of (shared) taxonomy terms, or if you maintain a WordPress network with a very large number of sites (especially if you have a custom routine for rolling out database updates), your review and testing of the upgrade routine would be much appreciated.

Booyah

The fun stuff on the taxonomy roadmap – like termmeta – depends on the complete elimination of shared terms. Let’s get the hard stuff done in 4.3, so that we can make all of your taxonomy dreams come true in the next couple releases.

#4-3, #dev-notes, #taxonomy