Completing the implementation of metadata registration with the REST API

A priority of the REST API team for the upcoming 5.0 release is to finally enhance WordPress metadata registration so that it can cover common API use-cases. This will help with adaptation of the REST API by making management of custom metadata in existing content endpoints easier, which should in turn also benefit any metadata-related work in Gutenberg. This post provides some insight into the surrounding issues, shows the current progress of the improvements, and highlights what still needs to be discussed.

A little history on register_meta()

The register_meta() function has been around since 3.3, but did not attract much attention with developers until WordPress 4.6 introduced enhancements to permit metadata registration for use within the REST API. As with other metadata functions, an object type (by core definitions either a “post”, “term”, “comment”, or “user”) needs to be passed to the function, alongside the meta key to register and the arguments for its behavior.

However, metadata in WordPress is often used only for a subset of one of the above four types. When dealing with post meta in a plugin for example, that post metadata commonly only applies to one or more specific post types – not to all posts of any post type. That is what currently makes register_meta( $object_type, $meta_key, $args ) unusable for the majority of REST API-related cases. Before the enhanced version was introduced in 4.6 many thoughts were put into this concern, with a subtype-aware implementation even being merged into core. That implementation was reverted before the release because it was unclear how to handle conflicts (more on that below). The ticket where all those changes were discussed is #35658.

Misconceptions about register_meta()

The improvements and their documentation caused developers to listen up, but unfortunately since the concept of an object type is rarely exposed in high-level WordPress APIs, many people misunderstood the first parameter as requiring a post type because post meta is the most common kind of metadata used by plugin developers. Some developers started using the function to register post type-specific meta keys without that function actually supporting that, as shown in this example:

Note:

This does not do what you may think it does:

register_meta( 'book', 'isbn', array( ... ) );

The code above will not register an ISBN number meta key for a “book” post type. It will register that key for a custom object type “book”—which probably doesn’t exist, unless a whole custom meta database table, a custom object database table, and surrounding setup has been implemented.

Those function calls fortunately don’t cause any harm in most cases since WordPress only contains four object types that use metadata (ignoring multisite). It becomes problematic though if the function is used to register metadata for the “post” post type.

Note:

This also does not do what you may think it does:

register_meta( 'post', 'some_meta_key', array( ... ) );

Since the “postpost type shares a name with the “postobject type, and since, again, the first parameter of register_meta() is for object type and not for a post type, register_meta() will register that meta key and arguments passed for all posts of any subtype (post type).

If you’re reading this and have used register_meta() in a way where you assumed it could register metadata for a single post type or taxonomy, at present we recommend you switch to using register_rest_field(), which is currently the only (however a little more involved) way to achieve this.

Adding subtype support for metadata behavior

A follow-up ticket #38323 aims to finally implement this missing functionality to register_meta() so that there’s an easy-to-use API to handle subtype-specific metadata in the REST API. It picks up the approach used before, where an additional object_subtype argument in the $args parameter can be used to specify the subtype the meta key should be registered with. Picking up the aforementioned example, you could register the ISBN for a “book” post type via the following code:

register_meta( 'post', 'isbn', array(
	'object_subtype' => 'book',
	...
) );

Open questions regarding subtype handling

Implementing meta registration for subtypes and modifying existing behavior to account for it is rather straightforward, and there is a patch available to experiment with the current approach. There are a couple things that need to be discussed though:

  • How should we deal with or prevent conflicts between meta keys of the same name and object type? The method currently allows to register metadata for an entire object type of course, so we cannot simply change that now and prevent those calls entirely. It appears we have four alternatives here:
    1. Allow meta keys of the same name and object type to be registered in any way. In case a meta key is registered for the current subtype, but also more generally for the object type, the behavior registered for the subtype takes precedence. Meta keys are internally stored nested under their object type and object subtype ($wp_meta_keys[ $object_type ][ $object_subtype ][ $meta_key ]). This is how the current patch on the ticket works, mainly because it is the most straightforward and flexible approach (it was also how the original code from #35658 worked). However it doesn’t deal with conflicts at all, so it is likely not preferable. An issue would be that on every update of a value, sanitization would happen for both behaviors, easily resulting in invalid data and unexpected errors. Throwing notices would be an option – but it would only help preventing conflicts, not actually prevent them.
    2. Allow meta keys of the same name and object type to be registered either per subtype OR for the entire object type. It would be based on whoever comes first takes priority. Meta keys are internally stored nested under their object type and object subtype ($wp_meta_keys[ $object_type ][ $object_subtype ][ $meta_key ]). If someone registers a meta key for an object type after someone else already registered the same meta key for a subtype of that object type, the process should be rejected and fail. If someone registers a meta key for an entire object type without the same meta key already being registered for a subtype of that object type, afterwards all meta keys for a subtype of that object type would be rejected. This prevents all possible conflicts since meta keys will only exist once per object type/object subtype combination, but still provides the flexibility of having multiple meta keys of the same name for different object subtypes. The implementation to prevent the conflicts during meta registration would be a little more involved though.
    3. Allow meta keys of the same name and object type to be registered only once in total. It would be based on whoever comes first takes priority. Meta keys are internally stored nested under their object type only, without an additional level for their object subtype ($wp_meta_keys[ $object_type ][ $meta_key ]). Whoever comes first, can register their meta key, every following request to add the same meta key for the same object type afterwards will be rejected. This prevents all possible conflicts since the data structure alone makes it impossible for two meta keys of the same name and object type to be registered. The approach generally appears to not be flexible enough: For example if two different post types were using a meta key of the same name, they wouldn’t be able to co-exist. However, based on the assumption that developers prefix all their metadata (which is probably not taught often enough), this option could very well work and be a straightforward solution.
    4. Allow meta keys of the same name and object type to be registered in any way, but use the behavior registered for the entire object type as a fallback only. In other words, if the same meta key exists with a subtype, only run that logic – for all subtypes where there is no specific behavior available, fall back to the behavior registered on the entire object type. To be precise, the difference between this approach and the first one is that in the first one the behavior (like sanitization and auth) would be executed for both, while here it only happens for the object subtype or the object type when no subtype-specific handling is registered. This is a conflict-free approach that at the same time doesn’t require any restrictions where calls to register_meta() would need to be rejected. The idea of using meta keys registered for an entire object type as fallback is slightly different from the other variants, but it seems to work well. It should still be highlighted that meta keys should be prefixed, but if that is a given, this implementation would even allow an individual plugin to provide more flexible logic for its metadata that may be needed for all post types (general behavior as fallback, subtype-specific behavior where needed).
  • Should we introduce easy-to-use wrapper functions like register_post_meta( $post_type, $meta_key, $args ) and register_term_meta( $taxonomy, $meta_key, $args ) which wrap register_meta() appropriately? The reason those might be useful is that the concept of object types and object subtypes is not commonly exposed in core’s documentation & high-level APIs (think about *_post_meta(), *_term_meta(), *_comment_meta() and *_user_meta() all wrapping the respective lower-level *_metadata() function). By adding these methods we would then recommend to only use those, and never recommend the use of register_meta() itself (as registering metadata for an entire object type is probably not desired). The only downside would be that comments and users don’t really have subtypes, so those two functions would either have a redundant first parameter or not have that parameter at all (which wouldn’t be future-proof).
  • A more specific issue: Should a comment’s comment_type value be considered a subtype? While one could argue that they are subtypes, they are treated entirely differently and do not really define how that comment works or how it can be edited. There’s also no flexible API around it to register additional comment types, and in the REST API there are no individual controllers per comment type. Currently, the patch ignores comment types, but this is still open for discussion.

Upcoming Meeting

The above items will be discussed in next week’s upcoming REST API meeting which will take place at the regular meeting time, Thursday, May 3rd, 2018 at 17:00 UTC in the #core-restapi channel. If you’re interested, please attend this meeting so that we can move forward with this ticket, as it should be included in WordPress 5.0. Also feel free to review the latest patch on #38323 and leave a comment on the ticket. While it is unlikely that this is the final implementation, a lot of it would not need to be adjusted regardless of the outcome for the above questions.

Extra credit goes to @kadamwhite for proofreading and several improvements.

#agenda, #rest-api, #team-update

Dev Chat Agenda: April 25th (4.9.6 week 4)

This is the agenda for the weekly dev meeting on April 25, 2018 at 20:00 UTC / April 25, 2018 at 20:00 UTC:

  • 4.9.6 planning
  • Updates from focus leads and component maintainers
  • Bub scrub communication
  • General announcements

If you have anything to propose to add to the agenda or specific items related to the above, please leave a comment below. See you there!

#4-9-6, #agenda, #core, #dev-chat

GDPR Compliance Chat Agenda – April 18

Agenda proposal:

  • A note on Privacy by Design and Cookies
  • Tickets to go out with WP 4.9.6
  • Erasure and export tickets (43546, 43551, 43602, 43637)
  • Owners for tickets
  • Open discussion

Join us on slack at 15:00 UTC.
Open trac tickets
#gdpr-compliance, #agenda

Dev Chat Agenda: April 18th (4.9.6 week 3)

This is the agenda for the weekly dev meeting on April 18, 2018 at 20:00 UTC / April 18, 2018 at 20:00 UTC:

  • 4.9.6 planning
  • Updates from focus leads and component maintainers
  • General announcements

If you have anything to propose to add to the agenda or specific items related to the above, please leave a comment below. See you there!

#4-9-6, #agenda, #core, #dev-chat

Dev Chat Agenda: April 11th (4.9.6 week 2)

This is the agenda for the weekly dev meeting on April 11, 2018 at 20:00 UTC / April 11, 2018 at 20:00 UTC:

  • 4.9.5 feedback
  • 4.9.6 planning
  • Updates from focus leads and component maintainers
  • General announcements

If you have anything to propose to add to the agenda or specific items related to the above, please leave a comment below. See you there!

#4-9-6, #agenda, #core, #dev-chat

GDPR Compliance Chat Agenda – April 11

Agenda proposal:

  • Announcements: Available texts and where to publish them
  • Plugin dev guidelines
  • Cookies
  • Trac tickets: Review of specific tickets
  • Open discussion

Join us on slack at 15:00 UTC.
Open trac tickets
#gdpr-compliance, #agenda

GDPR Compliance Chat Agenda – April 4

Agenda proposal:

  • Documentation: what texts do we need?
  • Marketing: How to announce the project to the world?
  • Trac tickets: Review of specific tickets
  • Open discussion

Join us on slack at 15:00 UTC.
Open trac tickets
#gdpr-compliance, #agenda

GDPR Compliance Chat Agenda – March 28

Agenda proposal:

  • Current status
  • Open discussion

Join us on slack at 15:00 UTC.
Open trac tickets
#gdpr-compliance, #agenda

Dev Chat Agenda: March 21st (4.9.5 week 7)

This is the agenda for the weekly dev meeting on March 21, 2018 at 21:00 UTC / March 21, 2018 at 21:00 UTC:

  • 4.9.5 planning
  • Updates from focus leads and component maintainers
  • General announcements

If you have anything to propose to add to the agenda or specific items related to the above, please leave a comment below. See you there!

#4-9-5, #agenda, #core, #dev-chat

GDPR Compliance Chat Agenda – March 21

Agenda proposal:

  • Right to erasure: What should the site owner be able to do? And the plugin author?
  • Other Trac tickets: Review of the other tickets
  • Open discussion

Join us on slack at 16:00 UTC.
#gdpr-compliance, #agenda