WordPress.org

Make WordPress Core

Tagged: options-meta Toggle Comment Threads | Keyboard Shortcuts

  • Jeremy Felt 6:27 pm on July 20, 2016 Permalink |
    Tags: , , options-meta   

    Additional register_meta() changes in 4.6 

    In the last 2 weeks, the direction of register_meta() has changed significantly from the original write-up. There was a meeting to discuss some of the changes and a recap of that discussion. Some other discussion after that meeting led to a much more simplified version of register_meta() that is now shipping in 4.6.

    Here’s what registering meta looked like in 4.5. This meta key has sanitization and authorization callbacks.

    register_meta( 'post', 'my_meta_key', 'sanitize_my_meta_key', 'authorize_my_meta_key' );
    

    The above code will continue to work in 4.6, though will not be considered completely registered. The callbacks will be registered, but the key will not be added to the global registry and register_meta() will return false.

    Here’s what registering meta looks like in 4.6. This meta key will have sanitization and authorization callbacks, and be registered as public for the WordPress REST API.

    $args = array(
        'sanitize_callback' => 'sanitize_my_meta_key',
        'auth_callback' => 'authorize_my_meta_key',
        'type' => 'string',
        'description' => 'My registered meta key',
        'single' => true,
        'show_in_rest' => true,
    );
    register_meta( 'post', 'my_meta_key', $args );
    

    The above will register the meta key properly and return true.

    There will no longer be a check for unique object types and subtypes for meta keys. There is no CURIE like syntax involved. Instead, be sure to uniquely prefix meta keys so that they do not conflict with others that may be registered with different arguments.

    Additional helper functions get_registered_metadata(), get_registered_meta_keys(), unregister_meta(), and registered_meta_key_exists() have been added to make the innards of the global data more accessible.

    The $wp_meta_keys variable should not be altered directly. It is possible that its structure will change in the future.

    Any code currently using register_meta() and expecting pre-4.6 behavior will continue to work as is. Please report any breaks in compatibility that might be found.

    For the full history, see #35658. 🙂

     

     

     
  • Jeremy Felt 10:02 pm on July 14, 2016 Permalink |
    Tags: , options-meta   

    register_meta() Discussion Recap (July 14, 2016) 

    As announced yesterday, there was a discussion today covering the future of register_meta(), something that has been in progress for WordPress 4.6 in #35658. This is a recap. 🙂

    Attendees: @helen, @ocean90, @rachelbaker, @mikeschinkel, @jsternberg, @sc0ttclark, @richardtape, @swissspidy, @joemcgill, @seancjones, @achbed, @jeremyfelt

    Chat log.

    Overview

    In #35658, register_meta() uses object subtypes when registering meta so that key registration can be considered unique.

    In 4.6 trunk, this information is passed as part of the 3rd argument—array( 'object_subtype' => 'books' )—to register_meta() and $object_subtype = '' has been added as an extra argument to several other pieces as a way to support that.

    After exploring a bit more, it’s clear that $object_subtype will continue to spread as an additional argument throughout many _meta() functions so that registered meta is handled properly.

    An alternative is to use CURIEs to describe complex meta types in a single string—'comment:reaction' rather than 'comment' and 'reaction'—so that the extra argument is not needed. Instead, processing exists to turn that string into an object type and subtype.

    Examples

    • post:post
    • post:books
    • term:category
    • user:user
    • comment:reaction

    The main question for today’s discussion

    Is CURIE like notation the right way forward for handling complex meta types?

    Decisions

    After a very good and thorough conversation about the above and other points, these are the decisions for moving forward with work on #35658:

    1. Meta keys will be registered using CURIE like syntax.
      • register_meta( 'post:book', 'isbn', $args );
    2. The : used in the string to register meta keys will also be used in filters.
      • sanitize_post:book_meta_isbn
    3. Core object types will fallback to default subtypes if one is not specified.
      • post becomes post:post, comment becomes comment:comment, user becomes user:user, and term becomes term:category.
    4. Meta key registration for all/any subtypes of an object type will not be included in 4.6, but is likely something to add in the future.

    Please check out the latest patches on #35658 and contribute code and thoughts on that ticket or share any questions/concerns in the discussion below.

    Thanks everyone for attending today!

     
  • Jeremy Felt 9:16 pm on July 13, 2016 Permalink |
    Tags: , , options-meta   

    register_meta() discussion on Thursday, July 14 

    Howdy!

    There will be a meeting in #core at Thursday July 14, 19:00 UTC to discuss the changes in register_meta() for the 4.6 release. The dev notes have been published, but there is still work to be done to ensure things are great.

    Of particular interest is a proposal to change to a CURIE like notation when working with the registration of meta. Instead of using two arguments (object type and subtype), there would be one argument containing both.

    As an example, If a post type was registered with books as the slug and ISBN was stored as sanitized meta:

    // In WordPress 4.5
    register_meta( 'post', 'isbn', 'my_sanitize_callback' );
    
    // In current trunk for 4.6
    register_meta( 'post', 'isbn', array( 'object_subtype' => 'books', 'sanitize_callback' => 'my_sanitize_callback' ) );
    
    // After proposed change
    register_meta( 'post:books', 'isbn', array( 'sanitize_callback' => 'my_sanitize_callback' ) );
    

    Please stop in #core at Thursday July 14, 19:00 UTC tomorrow if you are interested. If you can’t make it, please leave a comment on this post with your thoughts.

     
  • Jeremy Felt 8:30 pm on July 8, 2016 Permalink |
    Tags: , , , options-meta,   

    Enhancing register_meta() in 4.6 

    Note: Note: The direction of register_meta() has changed since this was published. Please see the most recent developer note explaining these changes.

    In 4.6, register_meta() expands to support the registration of meta keys and what to expect from those keys. See #35658 for the discussion around this change.

    The behavior of register_meta() is similar to register_post_type() in that the registration of this data is stored in the global scope. This makes an object’s meta data more accessible to parts of core and extending code.

    Why make this change

    In 4.5 and earlier, WordPress provided no method to explicitly register a meta key’s public state. Instead, register_meta() provided “protected” and “authenticated” meta. This can be used effectively for some things, particularly to determine what meta appears in the Custom Fields metabox when editing a post. It can not be used as a basis for arbitrarily showing meta keys and values to unauthenticated visitors.

    What does this change involve

    A global variable, $wp_meta_keys, contains all registered meta keys.

    The function signature of register_meta() has changed to support 3 arguments, the last being an array. That array should contain data about the meta with these key/values:

    • sanitize_callback, a callable method to provide sanitization. This is the new version of the current 3rd parameter in register_meta().
    • auth_callback, a callable method to provide authorization. This is the new version of the current 4th parameter in register_meta().
    • object_subtype, a string containing an object subtype slug. If there is no object subtype, meta will not be registered and a WP_Error will be returned instead.
    • type, a string indicating what type of data the meta value will be. This is intentionally not restricted to a specific list of data types, however full names should be used when possible. (e.g. boolean, integer)
    • description, a string containing a basic description of the meta value.
    • single, whether code retrieving meta for this key should expect a single or multiple values when using something like get_post_meta().
    • show_in_rest, whether this should be shown as part of a post’s meta endpoint in the WordPress REST API. Note that this should be treated as experimental until the WordPress REST API provides support for meta.

    The register_meta_args filter is available to add support for additional arguments. This need to explicitly whitelist further arguments is to reserve the right for core to add further arguments in the future; should you choose to use this filter, you should be prepared to follow along with further development. Your testing with trunk or betas is greatly appreciated!

    By default, only WordPress core object types (post, user, term, comment) can be registered. To add support for custom object types, use the wp_object_types filter. This whitelisting is similar to the above.

    Object sub types provide specific registration of meta keys to a type of data.

    • A standard WordPress post has an object type of “post” and an object subtype of “post”.
    • A custom post type registered with a slug of “my_cpt” has an object type of “post” and an object subtype of “my_cpt”.
    • A WordPress user has an object type of “user” and an object subtype of “user”.
    • A standard WordPress comment has an object type of “comment” and an object subtype of “comment”.
    • A standard WordPress category term has an object type of “term” and an object subtype of “category”.
    • A custom taxonomy registered with a slug of “my_tax” has an object type of “term” and an object subtype of “my_tax”.

    Meta keys must be explicitly registered for each object type and subtype combination.

    Additional helper functions get_registered_metadata(), get_registered_meta_keys(), unregister_meta(), and registered_meta_key_exists() have been added to make the innards of the global data more accessible.

    The $wp_meta_keys variable should not be altered directly. It is possible that its structure will change in the future.

    Any code currently using register_meta() and expecting pre-4.6 behavior will continue to work as is. Please report any breaks in compatibility that might be found.

    Example

    Here’s what registering meta looked like in 4.5. This meta key has sanitization and authorization callbacks.

    register_meta( 'post', 'my_meta_key', 'sanitize_my_meta_key', 'authorize_my_meta_key' );
    

    The above code will continue to work in 4.6, though will not be considered completely registered. The callbacks will be registered, but the key will not be added to the global registry. A WP_Error object will be returned with this explanation.

    Here’s what registering meta looks like in 4.6. This meta key will have sanitization and authorization callbacks, and be registered as public for the WordPress REST API.

    $args = array(
        'object_subtype' => 'post',
        'sanitize_callback' => 'sanitize_my_meta_key',
        'auth_callback' => 'authorize_my_meta_key',
        'type' => 'string',
        'description' => 'My registered meta key',
        'single' => true,
        'show_in_rest' => true,
    );
    register_meta( 'post', 'my_meta_key', $args );
    

    If you are currently using register_meta() and would like to maintain support for older versions of WordPress, the best method will be to check for the registration of the sanitization and authorization callbacks after calling register_meta() and then registering them manually if not present. This manual registration is all that register_meta() was previously doing. Continuing from the example above:

    // Pre-WordPress 4.6 compatibility
    if ( ! has_filter( 'sanitize_post_meta_my_meta_key' ) ) {
        add_filter( 'sanitize_post_meta_my_meta_key', 'sanitize_my_meta_key', 10, 4 );
    }
    
    if ( ! has_filter( 'auth_post_meta_my_meta_key' ) ) {
        add_filter( 'auth_post_meta_my_meta_key', 'authorize_my_meta_key', 10, 6 );
    }
    

    What’s next

    While the initial work on this is largely driven by the need to move the WordPress REST API forward, its scope is not limited to solving the problem of public access to metadata. By having information about the meta keys used in a code base, it becomes much easier for other pieces of code to make decisions about what can be done with this data.

    This data can one day be useful to everything from the Customizer, the editing experience for all object types, the Fields API, and many plugins that rely on metadata. Work will continue on transforming this $wp_meta_keys object and the methods surrounding it to be flexible and explicit. As WordPress core becomes familiar with register_meta() and more confident in the approach, the set of default arguments can and likely will expand to include more data about registered meta keys as required.

    Feedback

    Please leave any feedback you have on this new iteration of register_meta() in the comments below. Test it out in your plugins and themes to be sure it is working in a backwards compatible way. Try out the new function signature and register meta for objects and their subtypes. This continues to be a work in progress and feedback is important.

     
  • Eric Andrew Lewis 2:12 pm on May 3, 2016 Permalink |
    Tags: , options-meta,   

    A data schema for meta 

    register_meta() is a tiny function, which lets you register a sanitization and authorization callback for post meta, term meta, user meta or comment meta.

    We’re going to expand the utility of this function to describe the data type of the field. This will be useful for the REST API as well as the Fields API.

    This discussion has started in #35658, but I’d like to share the latest thoughts for general feedback.

    Imagine a “movie” custom post type which has a “star-rating” post meta field, which should accept the value 1, 2, 3, 4 and 5. REST API clients like the iOS app would want to know this information, so they can offer appropriate UI for their users. Ditto for the Fields API.

    Here are some loose thoughts on what that API might look like.

    Here is a post meta field for “mood” that accepts any string. The value would fit in a single row. There would only be one row per post, as it is not “repeatable.” Repeatable might be a default.

    <?php 
    register_meta( 'post', 'mood', array( 'schema' => array( 
    		'type' => 'string',
    		'repeatable' => false
    	)
    ) );
    

    Here is a field that consists of an array of URLs. The value would fit in a single row, perhaps in JSON format, i.e. ['some.url/','some.other/url'], and is limited to one row per post.

    <?php 
    register_meta( 'post', 'related_content', array(
    	'schema' => array( 
    		'type' => 'array',
    		'items' => array(
    			'type' => 'url',
    		),
    		'repeatable' => false
    	)
    ) );
    

    Here is a field that consists of an album as it relates to an artist and includes a number of properties. The value would fit in a single row, and multiple rows can exist for a single artist.

    <?php 
    register_meta( 'post', 'album', array(
    	'object_subtype' => 'artist',
    	'schema' => array( 
    		'type' => 'object',
    		'properties' => array(
    			'name' => array(
    				'type' => 'string'
    			),
    			'cover' => array(
    				'type' => 'wp-image'
    			),
    			'genre' => array(
    				'type' => 'string',
    				'options' => array( 'Rock', 'R&B', 'Shoegaze' ),
    			),
    			'release-date' => array(
    				'type' => 'date'
    			)
    		)
    		'repeatable' => true
    	)
    ) );

    What do you think?

     
    • Mark Howells-Mead 2:34 pm on May 3, 2016 Permalink | Log in to Reply

      That’s a good idea. However, PHP is type-less, so appropriate checks will need to be added when registering/writing/reading the field and its value.

      • Eric Andrew Lewis 3:00 pm on May 3, 2016 Permalink | Log in to Reply

        We’ll definitely validate the schema input upon registration to make sure it conforms to what we expect.

        Our first desire here is to expose the data types for the REST and Fields API. You bring up a good point: should we use the schema as a means for validating input from for example add_post_meta()? Curious what other folks think about this.

        • Scott Kingsley Clark 3:01 pm on May 3, 2016 Permalink | Log in to Reply

          add_post_meta suddenly returning WP_Error may be problematic, if we go that route. Not many people even check for add_post_meta / update_post_meta returns as it is though, to validate they even inserted/saved properly.

          I think whatever it is should be opt-in though.

    • Darren Ethier (nerrad) 2:57 pm on May 3, 2016 Permalink | Log in to Reply

      I like the idea, but I think to be properly implemented I think it’d be better that instead of some arbitrary string that describes the “type” you instead pass in a data type class name that describes the type and how its validated etc. This allows for MUCH more flexibility when it comes to plugins adding their own validation for how types etc get handled. There could be an interface that WP defines that all data-type objects should implement.

      Something like:

      interface WP_Data_Type {
      	/**
      	 * Name for the type ('object', 'string', 'stars') etc.
      	 * @return string
      	 */
      	public function name();
      
      	/**
      	 * How the datatype should be sanitized for the database.
      	 * @return mixed
      	 */
      	public function sanitize_for_db( $value );
      
      	/**
      	 * How the datatype should be sanitized for display.
      	 * @return mixed
      	 */
      	public function sanitize_for_display( $value );
      
      	/**
      	 * Is the value for the datatype valid?
      	 * @return bool
      	 */
      	public function validate( $value );
      }
      
      class WP_Integer_Data_Type implements WP_Data_Type {
      
      	public function name() {
      		return 'integer';
      	}
      
      	public function sanitize_for_db( $value ) {
      		return (int) $value;
      	}
      
      	public function sanitize_for_display( $value ) {
      		return (int) $value;
      	}
      
      	public function validate( $value ) {
      		return is_int( $value );
      	}
      }
      

      So WP Core could have some default data types that map from strings like ‘integer’, or ‘string’ etc. But plugins could implement their OWN special types that Core does not have like `My_Plugin_Stars_Data_Type` and core can use that for handling. One change that would need to be needed in the schema though is to have the plugin provide a `fallback_type` that is used if the plugin is inactive and core is not able to find the corresponding class for the name defined in `type`. So something like:

      register_meta( 'post', 'favorite-number', array( 'schema' => array(
      	'type' => 'My_Plugin_Stars_Data_type',
      	'fallback_type' => 'integer',
      	'repeatable' => false
      )
      ) );
      
      • Scott Kingsley Clark 3:06 pm on May 3, 2016 Permalink | Log in to Reply

        The method of storage provides for further expansion as full blown class objects, so this is just an interim solution until we can get Fields API into core.

      • Darren Ethier (nerrad) 3:41 pm on May 3, 2016 Permalink | Log in to Reply

        I totally glossed over the link to the related discussion in trac and that this is also based on the rest api schema for types. I still think my idea has merit but its kinda late to the party 🙂

    • Scott Kingsley Clark 3:03 pm on May 3, 2016 Permalink | Log in to Reply

      (sorry, moved my comment into the proper comment thread)

    • chatmandesign 3:24 pm on May 3, 2016 Permalink | Log in to Reply

      HELL YES!

    • Felix Arntz 5:36 pm on May 3, 2016 Permalink | Log in to Reply

      When I look at the third example, the value is an object with some properties. Does this also support an associative array? This doesn’t really matter for JS/JSON, but it does in PHP. Basically have `type` set to `array`, but at the same time support something like the `properties` key in the third example (although of course in an array it’s technically not a property).

    • davidperez 9:19 pm on May 3, 2016 Permalink | Log in to Reply

      Yes please! It’s a needed feature for meta simplicity…

    • Daniel Bachhuber 9:47 pm on May 3, 2016 Permalink | Log in to Reply

      Generally, I’m very supportive of adding more structured metadata around data in WordPress. I’m also not a huge fan of P2 discussion threads (because P2 kinda sucks), so keep that in mind if I forget to reply to a comment / don’t get an email notification about a comment.

      I think we should follow JSON Schema unless we make a deliberate, and intentional, reason not to. Following JSON Schema gives us access to all of the validation libraries already written, clients that can interpret it, etc. etc. Not following JSON Schema invents even more work for us, makes WP incompatible with existing tools that use JSON Schema, means we have to create a translation layer between this and the WP REST API, etc. etc. The mock code provided appears to mostly follow JSON Schema with a few unintentional (?) departures.

      More specifically, type should always be one of a fixed set of values. The wp-image type would be more correctly expressed as type=>integer,format=>wp-image. Or, we could define our own relation attribute.

      The options attribute has an equivalent in JSON Schema: enum

      Lastly, repeatable appears to be functionally equivalent to type=>array,items=>.... I suppose we could include it as a utility attribute, but doing so adds complexity.

      • Daniel Bachhuber 9:54 pm on May 3, 2016 Permalink | Log in to Reply

        Also worth mentioning: the problem of defining an application’s data models is one I’m sure many other CMSes, frameworks, etc. have worked on. It would be really good to apply their learnings and (try to) avoid their mistakes.

      • Weston Ruter 3:23 am on May 5, 2016 Permalink | Log in to Reply

        @danielbachhuber I agree. The definitions of non-scalar meta here are very similar to what is required of a schema to define what a widget instance looks like (#35574). While I don’t have a lot of experience with JSON Schema and some of the data structures seem somewhat complicated to describe data accurately, using JSON Schema as the standard format across WordPress and not just in the REST API makes a lot of sense to me.

    • chrishoward 2:42 am on May 4, 2016 Permalink | Log in to Reply

      Al Ay Lew Yah!! This is so desperately needed.

      Fields, as a bare minimum, need slug, label, creator, displayable and type. And then field meta to provide more detail.

      The creator is a way for a developer to uniquely identify their fields.. e.g. acf, woo-commerce, custom, etc…

      I develop plugins that allow users display content however they like and from from any sources. But when it comes to custom fields, all I’m able to present them with is a dropdown of field slug names (because WP mixes fields and other meta in the one).

      There’s little way to identify valid content ones as opposed to system ones. e.g. ACF creates both for each field. So an option to indicate if it’s a display field is valuable too.

      Furthermore, as Eric notes, there’s no way of knowing anything about how the field should be formatted. So 34.5 be an id in a library system, a single point numeric, a currency to two decimal places, or a

      Likewise, a date. What is this? 12/12/12 Is it a date? Is it maths – 12 divided by 12 divided by 12? Or is it some serial id system like model #, part #, year #?

      • chrishoward 2:45 am on May 4, 2016 Permalink | Log in to Reply

        oops! Don’t click reply til you mean it! 😛 UX fail. Need to add an edit option to these comments.

        Sorry for the missing format example where I just said “or a”. I meant to say a number like stocks or percentages that might be 3 or 4 decimals.

    • Ahmad Awais 3:06 am on May 4, 2016 Permalink | Log in to Reply

      Thus can be very helpful. I agree with Daniel as well that JSON schema might help us putting up a standard in olace that a number of libraries already use.

    • Julien 6:51 am on May 4, 2016 Permalink | Log in to Reply

      I’ll follow @Daniel Bachhuber. Also I have a question not directly related to this: why WordPress core is not embracing OOP development for their end-user API? Back from the first WCEU, a speaker asked the audience who has learnt to code thanks to WordPress and nearly 3/4 of the audience raise their hands. Would it be nice if WordPress could teach them gradually how to use classes and objects in place of nested arrays of nested arrays? WordPress is moving on its side while the main PHP community is standardizing its libraries in order to be compatible to any PHP projects and also re-usable.

    • Mike Schinkel 7:18 am on May 4, 2016 Permalink | Log in to Reply

      Are you seriously considering storing complex data in a single meta field as not just an edge case where you are storing a JOSN blog for convenience?

      IMO this is a really bad idea because it means you cannot use SQL to query the data. Let me beg the team to consider storing complex data like this. It is a format that really stood the test of time and many different websites, and it can handle any level of complexity. It can also support a generic mention of converting too and from a JSON blob:

      https://bitbucket.org/newclarity/sunrise-1#markdown-header-post-meta-storage-format

      ALSO, I would strongly urge against adding nested arrays. It make for brittle code that can easily be mistyped and it makes it impossible to do a generic wp_parse_args() in order to merge in default attributes. If you MUST use nested attributes please consider using nested keys instead with colons to seperate, like so; it is easily to extract the subarrays from these nested keys when you need it but this supports wp_parse_args() much better, and in my experience you really need that:

      
      register_post_meta(  'album', array(
      	'object_subtype' => 'artist',
      	'schema:type' => 'object',
      	'schema:properties:name:type' => 'string',
      	'schema:properties:genre:type' => 'string',
      	'schema:properties:genre:options' => array( 'Rock', 'R&B', 'Shoegaze' ),
      	'schema:properties:release-date:type' => 'date',
      	'schema:repeatable' => true
      ) );
      

      But even that seems over-architected but I am really struggling to understand the example — the semantics of the example are not clear to me — so I can’t give a simplified example.

      But I do feel this area is critical to get right because getting it wrong dooms many people to writing too-complex and rather error prone code.

      • Weston Ruter 3:27 am on May 5, 2016 Permalink | Log in to Reply

        @mikeschinkel Storing complex data in single meta values (usually serialized data) are a fact of life in WordPress. Widgets are perhaps the most standard example of this. They invariably require storage as complex data in single values. I’ve proposed in #35574 that JSON Schema be used to describe widgets so that they can be exposed via the REST API.

        As for JSON Schema the format, since it is a standard(ish) format there should be ample tools to validate that a JSON Schema written is valid, and also to auto-generate a Schema definition based on example data being provided.

        • Mike Schinkel 4:17 am on May 5, 2016 Permalink | Log in to Reply

          Hi Weston,

          Thanks for replying to the comment.

          > _”Storing complex data in single meta values (usually serialized data) are a fact of life in WordPress. Widgets are perhaps the most standard example of this. They invariably require storage as complex data in single values.”_

          I should have taken more care to be clear on that. You are correct, in some cases. WordPress definitely stores PHP serialized data into `wp_options` for things like widgets, active plugins, cron, sidebars, user roles and more. And I absolutely agree that is a best practice.

          WordPress also stores `wp_capabilities` and `session_tokens` in `wp_usermeta`, and while I don’t love that it does it is not horrible unless you start getting 10s or 100s of thousands of users.

          But where WordPress generally does not store serialized data is in `wp_postmeta`, and I think it would be a huge mistake to make it easy for developer to start doing so as it could several impact scalability when it comes to 10s or 100s of thousands of posts, or even millions of posts. Which is not terribly uncommon as more large media companies start to use WordPress for their CMS, unless of course they offload actually serving JSON content to another server, for example.

          > _As for JSON Schema the format, since it is a standard(ish) format there should be ample tools to validate that a JSON Schema written is valid, and also to auto-generate a Schema definition based on example data being provided._

          Nothing I am suggesting would conflict with that goal.

          @e can easily give you exactly what you need without resorting to serialized data in `wp_postmeta` nor `wp_termmeta` (although that would be less problematic.)

          If you want to be able to have a function `get_postmeta($post_id,’album’)` to be able return an object so you can pass to `json_encode()` we can easily write a serializer/deserializer that stores data using meta_keys in the following format:

          _person[name]	 => Mike Schinkel
          _person[gender] => 'male'
          _person[links][twitter] => 'http://twitter.com/mikeschinkel',
          _person[links][2] => 'http://facebook.com/mikeschinkel'
          _person[links][3][linkedin] => 'http://linkedin.com/in/mikeschinkel'
          

          We have used this storage format for years on client projects to numerous to count and it has never once failed us.

          I will be happy to write the serializer/deserializer because I have pretty much already written it.

          • Mike Schinkel 4:21 am on May 5, 2016 Permalink | Log in to Reply

            Arrrgh, I really wish I could edit my posts. Here is the code fixed:

            _person[name]	 => Mike Schinkel
            _person[gender] => 'male'
            _person[links][twitter] => 'http://twitter.com/mikeschinkel',
            _person[links][2] => 'http://facebook.com/mikeschinkel'
            _person[links][3][linkedin] => 'http://linkedin.com/in/mikeschinkel'
            

    • schlessera 9:59 am on May 4, 2016 Permalink | Log in to Reply

      Absolutely agree that different meta values need more structure. However, when looking at the example provided, I’m a bit perplexed. This looks like you’re trying to cram a CMS into the meta fields of the existing CMS. If you want to have separate, complex entities, there’s already CPTs & taxonomies, etc… for that. I don’t understand why you would want to save an album with all of its data within one single meta field… Why not an `album` CPT, and store the ID in the meta?

      • Eric Andrew Lewis 11:38 am on May 4, 2016 Permalink | Log in to Reply

        I considered adding a caveat to that example “Not that you should do this, but you could,” as it would make more sense in your content structure. There is probably a better example to show key-value based entity support.

        • Mike Schinkel 4:23 am on May 5, 2016 Permalink | Log in to Reply

          My experience as a programming trainer is that any code you publish will get copied and widely used in production code.

          So if they should not do it I would heartily recommend that you don’t ever show it. Or you will be horrified later as you see your example code in someone’s plugin, copied verbatim from your example (I know, I’ve been there.)

    • Franz Josef Kaiser 11:08 am on May 4, 2016 Permalink | Log in to Reply

      I am stunned by that proposal. This is maybe the most (backwards compatibility) breaking change in WP history. The current purpose of the function is completely different:

      register_meta( $type, $key, $sanitizeCb, $authCb )

      Summary of current status:

      • Name a meta type – mapped together with key to group and standardize sanitization callbacks on typed (user, post, etc.) fields
      • Name a meta key – mapped to the DB
      • Map a meta key to a type of data object (post, user, etc.)
      • Decide if a meta field is “protected”, meaning if it shows up in the Custom fields meta box, or if the view is handled by some meta box *)
      • Decide who has access (based on capability, role name, other things like other meta data) to a custom fields attached DB operations
      • Give access to meta data that was registered by a plugin and needs to be accessible again when the plugins is not present anymore – via the Custom fields meta box and an intermediate plugin

      To address this:

      We’re going to expand the utility of this function to describe the data type of the field.

      This is what the function currently does. The proposal on the opposite tries to:

      • Describe how many meta values for a single key can exist – Handled by the implementation
      • How the data is structured – Handled by sanitization callback and input views
      • Describe how multiple values, or the serialized input is structured – Again, handled by sanitization callbacks

      There are more serious issues that need to get addressed:

      • Meta fields starting with an underscore/ _ are “protected” per default and therefore hidden from the UI, the Custom Fields metabox. This makes them inaccessible when a plugin is deactivated and custom intermediate plugins need to be written. The is_protected_meta() should throw a _doing_it_wrong() warning and suggest using __return_false for the $auth_callback
      • Meta key names can be anything, even CamelCase. This should get standardized so mistyping is harder. They should get checked by the following Regex: "^[a-z0-9_]+$"
      • Disallow the use of registered post type names as this locks post meta data down and disallows switching them to a new post type – portability
      • Drop conjunctions (with), articles (the) and other filler words from the meta key name. Only allow boolean flags (example: has_*, is_*) as this is unavoidable to cover different context.

      I really, really do not understand why this function should be completely repurposed. What imo is missing is a central registry that validates against the DB to show orphaned meta keys and that can be used to map DB data to business logic. I also think that a central registry would be needed, but proposing to make a cat bark? No, thank you. And if everyone really thinks they do need an adapter or decorator for the Fields and the REST API in core, then please break that out into something completely new.

      • Eric Andrew Lewis 11:56 am on May 4, 2016 Permalink | Log in to Reply

        > This is maybe the most (backwards compatibility) breaking change in WP history

        This would not be a breaking change, sorry if that was unclear. We would support the function being called as it is now, without the schema information.

        > Disallow the use of registered post type names as this locks post meta data down

        This would be good to consider, as other APIs in WordPress break when using reserved terms.

    • Anton Timmermans 10:37 am on May 5, 2016 Permalink | Log in to Reply

      I don’t think it is a good idea to repurpose the `register_meta` function to do something else. It is clearer and a better practice to introduce a new method and if the old meta is now undesirable to deprecate it. The functions in WordPress are very complicated a lot of the time which is partly caused by having multiple responsibilities in one function.

      I am in full support of following a broader standard here like JSON schema. This makes us more interoperable with other systems, which is a good thing.

    • Weston Ruter 4:02 am on May 10, 2016 Permalink | Log in to Reply

    • Mark Root-Wiley 5:26 pm on June 6, 2016 Permalink | Log in to Reply

      I was thinking about custom search in WordPress this weekend and realized this project might consider adding an `exclude_from_search` parameter a la `register_post_type()`. This would then allow search plugins to automatically index meta and maybe even lead to WordPress core search doing the same down the road.

  • Scott Kingsley Clark 3:10 pm on March 14, 2016 Permalink
    Tags: , , options-meta   

    Fields API — Where we’re at 

    Over the past few months, we’ve been working towards one of the most wide-reaching API’s to be suggested for core inclusion, apart from the REST API. It has the potential to touch nearly every area of WordPress that deals with forms and fields. The Fields API is nearly there, and we’re seeking more feedback on a number of areas we need to tie down before we go into the official proposal stage for core inclusion.

    What is the Fields API?

    We just published a specification for the Fields API last week and we’re seeking constructive feedback on that now. Please take a moment to read about it, ask questions, and voice your concerns with us so we can address those things before we go to the next official stage.

    There’s also some helpful docs which include TerminologyRegistration of fields, and example code.

    What can you do with the Fields API right now?

    We’ve put together a number of implementations to show the potential of what the Fields API can accomplish inside of core. Each implementation is an example of what could end up in WordPress core, should the Fields API itself be merged.

    • Edit User screen (@sc0ttkclark, @technosailor)
      • Our internal implementation of the Edit User screen replaces core’s entirely using the Fields API.
      • Here’s a code example to add more fields (and sections and controls) to the Edit User screen
      • Backwards compatibility hooks
      • To-do: Further custom markup and/or styling to match uniform layout
    • Edit Post screen (@sc0ttkclark, @brentvr)
      • Our internal implementation of the Edit Post screen will replace core’s entirely using the Fields API, we only have support for adding new sections / controls right now
      • To-do: Register sections / controls based on what’s on the Post Editor screen currently (@brentvr is working on this now)
      • To-do: Add additional output types (no meta box, etc)
      • To-do: Backwards compatibility hooks
    • Term Add / Edit (@sc0ttkclark, @technosailor)
      • Our internal implementations of the Add Term and Edit Term screen replaces core’s entirely using the Fields API
      • Backwards compatibility hooks
    • Comment Edit (@sc0ttkclark)
      • Our internal implementation of the Edit Comment screen will replace core’s entirely using the Fields API, we only have support for adding new sections / controls right now
      • To-do: Register sections / controls based on what’s on the Comment Editor screen currently
      • To-do: Add additional output types (no meta box, etc)
      • To-do: Backwards compatibility hooks
    • Settings Pages (@sc0ttkclark, @technosailor)
    • Widget Forms (@nicholasio, @sc0ttkclark)
      • Our internal implementation of the Widget forms will integrate with WP_Widget to allow form fields to be registered for widgets and rendered/saved automatically without the custom code every widget ends up needing currently.
      • To-do: WP_Widget integration (@nicholasio is working on this now)
    • Nav Menu Items (@diddledan, @sc0ttkclark)
      • Our internal implementation of the Nav Menu Items overrides the core Walker class used for building the Nav Menu Item forms, and uses the Fields API to output registered controls.
      • To-do: More compatibility work for CSS
      • To-do: Look into Customizer integration
      • To-do: Getting/saving values for nav menu item fields
    • Media
      • We do not currently have an implementation for this yet, but one was proposed by @wpsmith, and we are seeking additional help from anyone looking to make this happen
      • To-do: Add sections and controls to Media Edit
      • To-do: Add sections and controls to Media Modal Add/Edit
    • REST API
      • We currently have limited direct integration with the REST API, but we’d like to work with the REST API team towards implementing REST API options and building in addition configurations the REST API can consume for it’s endpoint(s)
      • register_rest_field integration

    Possible Roadmap to Merge

    Here’s an example of how Fields API could eventually be merged into core, along with other implementations.

    • Release 1: Merge Fields API into core
    • Release 2: Merge Implementation 1 into core
    • etc..

    So each release could focus on a different implementation, which of course is a great time to coincide to UX / design work on each area, like the User Profile screen. There’s lots of options here, but hey, let’s discuss it!

    How to help

    If you’d like to help anywhere, please just reach out and I’ll get you into it. Every area needs review, some areas still need more work to finish, but any area could use additional help.

    So who do you contact or where do you go to help?

    Next Meeting Agenda – March 14th, 2016

    Our next meeting is Monday 20:00 UTC 2015 and it promises to be eventful. We plan on discussing a number of things including:

     
    • mrpritchett 3:16 pm on March 14, 2016 Permalink | Log in to Reply

      Great job Scott and team! Super excited about this! I guess my only question is if the REST API is integrated completely before this, would integration of the REST API with the Fields API not be a requirement to merging?

      • Scott Kingsley Clark 3:18 pm on March 14, 2016 Permalink | Log in to Reply

        Good question, because both aren’t fully implemented yet (REST API meta endpoints, etc), that question remains unanswered until we get some feedback and direction from the movers and shakers I suppose. 🙂

    • Weston Ruter 4:00 pm on March 14, 2016 Permalink | Log in to Reply

      @sc0ttkclark re: Widget Forms, this integration would be closely related to #35574. If the REST API JSON schema were rich enough, there could be a default WP_Widget::form() implementation that just output fields based on the associated schema. But since it isn’t, I understand this is where the Fields API will come into play. But there seems to be some key overlap.

      • Scott Kingsley Clark 4:02 pm on March 14, 2016 Permalink | Log in to Reply

        Certainly! I’d love to see the Fields API enable this ability in the REST API, I think it’s a natural fit and plays into other things we’re planning to do to give REST API the information it seeks about these configurations.

      • Scott Kingsley Clark 4:04 pm on March 14, 2016 Permalink | Log in to Reply

        I definitely think a method that defines the widget’s properties in more detail would be useful, which is where Fields API could be utilized too. So I think overlap can be reduced if we hold on #35574 for Fields API to play a part.

    • Ahmad Awais 6:32 am on March 15, 2016 Permalink | Log in to Reply

      Incredible job, Scott and all the contributors. I am looking forward to this project and almost feel guilty for not contributing.

    • Ryan Boren 3:08 pm on March 15, 2016 Permalink | Log in to Reply

      How about some screenshots of the implementations?

  • Scott Kingsley Clark 11:31 pm on January 12, 2016 Permalink
    Tags: , , options-meta   

    Fields API updates from Q4 of 2015 

    It’s been a while since I updated everyone here about progress on the Fields API. I’ve been hard at work in the code and haven’t come up for air until now, but I’m planning on more regular updates in the coming weeks 🙂

    Lots of progress since our last chat summary in early October, here’s a bird’s eye view of what we’ve been up to:

    Where can we use help right now?

    • Review our User Profile screen implementation, let us know if it makes sense to you as a developer. It’s the kind of code you’ll be looking at needing for any custom screens you want to implement in plugins or themes, and it’s the kind of code we’re looking at using for other WordPress screens too (Post editor screen, Term editor screen, etc..)
    • Help us create use-cases for each type of object (Custom Post Types, Custom Taxonomies, Users, Comments, Media, Widgets, Nav Menu Items, etc..) — we’re specifically looking for what custom fields you could create for these use-cases and how it could make sense in real world scenarios. You can start out with just the structure, we can help you build the code for it.
    • We’re getting ready to release this on WordPress.org as a plugin, we could use some help to write up a good readme and ensure it directs people to the correct places for involvement

    So who do you contact or where do you go to help?

    Oh and did I mention that every minute you help us, you make someone really really really happy in the future? It could be your future self that gets to reap the rewards of your hard work!

     
    • eddr 12:47 am on January 13, 2016 Permalink | Log in to Reply

      Hi Scott – great stuff. That’s a good feeling to see all the people and community involved

      Like many, I’ve been using custom fields intensively and probably one thing that comes to mind for some is the database structure / efficiency. Querying meta fields/values is kind of a disaster for big sites with complex data. I know that most implementations create dedicated tables, but still – any thoughts about it?

      • Scott Kingsley Clark 5:10 pm on January 13, 2016 Permalink | Log in to Reply

        A few optimizations I do on bigger projects:

        • Separate large sets of data not needed across a site, but only within a specialized section of the site into it’s own multisite blog tables
        • Add post_type column to wp_postmeta, add more indexes to wp_postmeta (post_id, meta_key(30), meta_value(30); and other combinations you may want), use custom SQL queries to do meta lookups to SELECT post_id where the meta_key + meta_value match for post_type; and when inserting your meta via add_post_meta, take the return value of $meta_id and update post_type to the correct value)
        • Using a plugin like Pods, I can extend a post type to be a table for specific custom fields, this allows me to add indexes to that table and do more complicated lookups more easily without use of CAST() or LONGTEXT on custom field data (correctly have them MySQL-typed)
        • Use ElasticSearch / ElasticPress (or any other ElasticSearch plugin for WP) that allows you to do meta lookups more quickly than expensive SQL queries in WP_Query, some plugins like EP even integrate with WP_Query.
        • Kochhase 6:39 pm on January 13, 2016 Permalink | Log in to Reply

          Using a plugin like Pods, I can extend a post type to be a table for specific custom fields, this allows me to add indexes to that table and do more complicated lookups more easily without use of CAST() or LONGTEXT on custom field data (correctly have them MySQL-typed)

          Hi Scott, why do we have not pods in core? Many people need a tool to extend wordpress to an CMS as drupal. In combination with the field api this would make sence.

          I hope for that in WP 4.6

          regards Henryk

          • Scott Kingsley Clark 6:41 pm on January 13, 2016 Permalink | Log in to Reply

            Some things don’t belong in WP core, and that’s alright. WP can’t be everything to everyone. Plus Pods does a whole lot more than just that one thing 🙂

    • Ahmad Awais 7:57 am on January 13, 2016 Permalink | Log in to Reply

      Hey, Scott!

      I was thinking, what if we could have mini customizer instances?
      E.g. I created a form and I want to design it different from all other forms. In theory, I’d click a Customize button that will redirect me to the front end with customizer instance (only with the options that are needed to design that particular form). After I am done designing it, I’ll get to save it and get back to where I was in the form plugin.

      Right now, this UX and flow doesn’t exist. Customizer’s class is defined as final, there is not much we can do. But with Fields API, what if instead of creating forms inside widgets, we could get such mini customizer instances?

      Did it make any sense at all? I hope it does.

    • leemon 10:08 am on January 13, 2016 Permalink | Log in to Reply

      Great to see this moving forward! Thanks!

    • Hans-Helge Buerger 11:00 am on January 13, 2016 Permalink | Log in to Reply

      Hey Scott, I just did a quick read up about the Field API b/c I didn’t know what it is or what it’s about. Great stuff. I’m looking forward to see this in core.

    • Najum Rahim 12:44 pm on January 13, 2016 Permalink | Log in to Reply

      Ahmad,You have great idea

    • Marcus 5:03 pm on January 13, 2016 Permalink | Log in to Reply

      Looking forward to seeing the plugin. The specification and structure looks good to me at a glance.

      r.e. use cases, what about a simple skeleton plugin/snippet creating some generic set of custom fields, so we can modify that? for example, users:

      section – address

      • address 1 (text)
      • address 2 (text)
      • town/city (text)
      • county/state (text)
      • Zip (text)
      • country (DDM)

      you could go further and showcase every field type, maybe ditch a use case like the above and just create fields named literally e.g.

      section – custom form

      • text field
      • textarea
      • checkbox
      • etc.
    • Tran Ngoc Tuan Anh 8:58 am on January 22, 2016 Permalink | Log in to Reply

      Very nice idea. From years working on Meta Box plugin, I see the APIs are very different when implement fields for posts, terms, settings pages. It’s really helpful when everything is merged into 1 common API.

      I also want WordPress does this in core, e.g no another layer of abstraction code. Right now, plugins like Meta Box, ACF, Pods already add a layer for users/developers to work with custom fields. I just don’t want more layer in between.

  • Scott Kingsley Clark 7:29 am on October 8, 2015 Permalink
    Tags: , , , , options-meta   

    Fields API chat summary – October 5th, 2015 

    Present: @sc0ttkclark, @nicholas_io, @tomharrigan, @ericlewis, @potatomaster

    Logs: https://wordpress.slack.com/archives/core-fields/s1444021200000000

    • I just got 100 hours from 10up to work on Fields API!
    • I will be working on getting the WP 4.3 Customizer changes put into the Fields API, my first pass doesn’t have unit tests passing yet
    • We’ll be fleshing out Control classes, based on Customizer control classes and expand the main control class into individual classes as opposed to a ‘switch’
    • We laid out a few implementations we’d like to get into prototyping
      • User profile fields (piggy backing existing UI of section heading + fields) @sc0ttkclark
      • Settings API (cue the oooh’s and aaah’s sound effect) @sc0ttkclark
      • Post editor (meta boxes + fields) @tomharrigan
      • Widget forms @nicholas_io
      • Future: Term editor (sections + fields)
      • Future: Comment forms?
    • We want to improve the main Fields API readme to better explain the project, offer more links to information about the Customizer API since it’s what we based the Fields API on, and flesh out more examples
    • We need more examples, so any use-cases we can put together for any object type, would be handy to start putting that code together (structures, not custom implementations or overrides)

    We certainly could use additional contributors involved with the project, especially as we seek to start more implementation prototypes of how things could work. Just hop into Slack #core-fields or check out our Github repo. Over the next 5 weeks my involvement in the project will be greatly increased, so if you are going to get involved — now would be the right timing!

    Next chat: Monday 20:00 UTC 2015 (every Monday)

     
  • Scott Kingsley Clark 4:15 pm on August 3, 2015 Permalink
    Tags: , options-meta   

    Fields API: Request for review of direction 

    Over the past many months this year, I have been working with guidance from @helen on the new Fields API with the intention of inclusion into WordPress core. It’s based on the Customizer API, so those who have experience with it will see a lot of similarities. The goal isn’t to create a duplicate API though, and in our proposal we would be implementing the Fields API within the Customizer API.

    What does that bring to core as a whole? It gives core a very real and far reaching API to add fields to any type of object in a standard interface. Given the growth of the Customizer API and its inclusion of fields, @helen and I both agreed it’d be great to leverage everything it can do in our efforts here. The goal isn’t focused on the actual Customizer UI or editing on the front-end, it’s all data-oriented.

    We would create implementations behind the scenes in the various existing APIs such as the Customizer API (now) and Settings API (later) while maintaining backwards compatibility. We’re also hoping to implement the Fields API within the User profile screen. This would give core more flexibility towards revamping the User profile screen in the future, without having to tackle even more than just a UI refresh. That’s also not mentioning the fact that plugin and theme authors could leverage this API to extend WordPress with their own fields in the various areas. Future places we’d look at integrating Fields API with would be the Post editor, which I’ve pegged as our third focus, behind User profile fields and the Customizer API.

    Anyways, that leads us to the point we’re at now, we have an API but we don’t have all of the built-in Field Types (control types) yet or the latest Customizer API changes from 4.3 merged upstream into the Fields API. There are unit tests that are running (and passing!) so that is going to help us on our road towards core for sure.

    We need developers to give their feedback on the direction we’re heading. Here are the relevant links for those reviewing:

    I’d love more feedback from anyone contributing to WordPress, or the *countless* plugin and theme developers who would use the Fields API. Please comment on this post, or if you’d like to discuss further, you can hop into our weekly meetings on Mondays (Monday 20:00 UTC 2015) in the WP Slack #core-fields channel, or ping me there anytime.

    Update – 08/10/2015:

    We now also have a growing list of developers who are behind this project and would implement it in their projects if it were merged into core.

     
    • Matt Gibbs 4:48 pm on August 3, 2015 Permalink | Log in to Reply

      Thanks for all the hard work, Scott and Helen! The specifications doc looks very detailed. I’d like to see more usage examples though. Similar to the one on https://make.wordpress.org/core/2015/05/27/metadata-api-project-reborn-the-new-fields-api-project/

      1. How to create custom field types (or Controls as its called now)?
      2. Does a field require a section?
      3. What happens if the field capability isn’t set? Does it fallback to the parent section / screen?
      4. How are repeatable fields handled in the current spec?
      5. Is the field API itself concerned about field order? Should it be?
      6. How granular are field sections? E.g. can a field be added to a single post item (or user) only?

      • Scott Kingsley Clark 5:55 pm on August 3, 2015 Permalink | Log in to Reply

        1. We’re still ironing that part out, control types / field types is the next area of the spec we have to nail down. Feel free to join us on Mondays for our weekly meetings if you want to help us get that part finished.

        2. Currently, it functions the same as the Customizer API, which allows you to add as many fields as you want, without sections or screens (panels). The differentiation is when you add a control, you specify the section or screen you want to add it to.

        3. Fields don’t require capabilities, controls don’t either. We handle the same capabilities checks as the Customizer API does right now — https://core.trac.wordpress.org/browser/trunk/src/wp-includes/class-wp-customize-setting.php#L439 — which is to say, if you don’t provide a capability check, it is wide open. We should nail this down further and I think it’d be good to default to no-access. We need dev input on that.

        4. There is no preparations for repeatable fields or repeatable groups at the moment, that’s something we want to nail down in the control types / field types discussion, as there may be things we want to do there. The existing Fields API spec is almost entirely based off of the Customizer API and no additional functionality has been added for that yet. It’s on our list though, we want to ensure any effort we put forth into core does not limit developers or the common ‘loop field’ use-cases. Whether ‘loop fields’ go into core or not, that’s not something I can speak to at this point.

        5. Yes, the spec makes use of the ‘priority’ option for screens, sections, and controls. Works just like filter/action priorities do.

        6. Screens, sections, controls, and fields are all added to a specific object type / name (for example: post type : post). Further definition based on a conditional like a specific ID, capability, etc, could all be handled through an additional callback or filter that we can add inside of the check_capability methods for screens / sections / fields.

    • Brent Jett 8:23 pm on August 3, 2015 Permalink | Log in to Reply

      Cool spec! I like the idea of a broad-reaching common syntax for adding fields from themes/plugins, but I’m wondering if there isn’t a more modern/human-readable format or syntax for this. My immediate thought was JSON files (OOP code underneath obviously). WordPress exposed object oriented API so late in it’s relative lifespan that OOP code like the customizer API still looks very foreign to themers. Given that most app platforms now use JSON manifest files to declare things like configuration, it would seem to me very natural for WP to look at adding a more approachable syntax like that to it’s theme configuration APIs.
      But I could be way off. Nice work either way.

    • Mike Nelson 10:14 pm on August 3, 2015 Permalink | Log in to Reply

      In Event Espresso we created a similar system called the “Forms System”, inspired primarily by Django’s forms. EE Forms basically define how to display an HTML form, how to validate the user-provided input on both the client-and-server side, and how to sanitize/normalize the data for use in the PHP code. (It leaves the job of saving data to either our models system, wp options api, and sometimes direct wpdb).
      EE “form sections” are roughly equivalent to your “screens” and “sections”, and EE “inputs” are roughly equivalent to your “controls”.
      Some differences that might spark some ideas for you:

      1. EE form-sections are nestable; so you can have sub-sub-sub-sections etc. This has allowed for more re-use. Eg, say you wanted to have screen section for multiple post custom fields: the “custom fields” would be a section, and then it would have a different sub-section for each post custom field.
      2. Each form section can have its own “layout strategy”, a separate class that dictates how to layout the form section’s inputs in the HTML, and its sub-form-sections. So if you want to layout your form’s inputs in a very standard way, you’d select one of the pre-set layout strategies.
      3. If you want to layout your forms input in the HTML in a very custom way, template code can be passed a form section which gives the template code access to the sub-sections and inputs and the template code can dictate where to place what
      4. sub-sections can actually just be chunks of HTML, “html subsections”. These have been good for extra stuff that’s important for the display of the form, but aren’t necessarily related to form-data
      5. the EE forms system is separate from our “model system”, although they’re sometimes related. The forms system is primarily focused on displaying HTML forms, and then validating and sanitizing the user’s input; whereas the models system focused on defining the underlying data stored in the database, how that data is related to other data, how to save and retrieve that data, etc. We have a form section base type that can automatically create sub-sections and inputs based on a particular model.

      Other than that, it’s really quite similar.

      Also it’s not that clear: is the fields API intended for use in the frontend too?
      Also it seems if this is intended primarily for defining the HTML forms and how they get validated/sanitized. I think having a separate API for defining the underlying data would be beneficial, in cases where you don’t want every attribute of the data to necessarily be represented in HTML forms, or you might want to provide different forms for editing the same underlying data.

      • Scott Kingsley Clark 4:16 pm on August 4, 2015 Permalink | Log in to Reply

        1. I’d like to explore the idea of sub-sections and sub-fields (sections that can have sections in them, fields that can have fields in them) in our class architecture. Even if they’re not utilized in core, the ability to use them or for plugins to enable their use would be a big thing.

        2. We’re looking at something similar, though not really, it’s mainly that each screen will have it’s own fields display and saving implementation, much like the Customizer itself has one.

        3. There are filters and actions in place for this right now in the Customizer and the Fields API

        4. I don’t doubt someone could extend the Fields API with a new control type or a custom section class that would be able to output just HTML as set in the definition options. That could be useful for some cases.

        5. Fields API is based on the Customizer API, it’s somewhat split up between modeling and display, but it’s not entirely MVC at the moment.

        6. You can use the Fields API anywhere, you would write your own implementation to display fields and process the $_POST data accordingly, utilizing the underlying Fields API to access the data structures and field types input HTML etc. You could even use the Fields API for any object type you want to build into it, there are hooks in place that support custom object types, for purposes of plugin developers to utilize the Fields API for their own things (Gravity Forms, Pods, NextGen Galleries, etc).

        7. Each implementation can override the core output, to do whatever it wishes. For instance, Customizer API supports previews and makes heavy use of JS to do what it does. Our Customizer API implementation extends the Fields API but does not limit the ability for those things, and many others for other implementation needs.

    • thomask 11:02 pm on August 3, 2015 Permalink | Log in to Reply

      I am really looking forwards to this, actually for me this is the second most crucial missing feature for WP – the first is multilanguage support. I use Advanced Custom Fields for almost every web i do, and however it is very nicely written and easy to use, i would prefer some build-in support.
      I have read it all just briefly and will have to try it, from my experience what is important is
      1. support for multilanguage fields (so that i can have common field for all languages version of one post, as well as translatable field) – but this is i guess more question on the multilanguage plugin (or future WP multilanguage support)
      2. text field (just simple text), term fields and post field (N:N relation field), wysiwyg field, user selection field … and/or easy way how to create new field types with standardised look, values control etc.
      3. allowed values control and definition, read only possibility, required field possibility
      4. possibility to add fields also to terms (!!!) – imo terms (edit-tags.php) are currently very stupidly different from the posts – it should have the same look, so that we could use the same fields and functions.
      5. easy definition of post types and other conditions, that are required for field to be displayed
      6. advanced feature, but very usefull in many cases – possibility to set-up condition for field, that should be calculated live on the post screen depending on some live changes – e.g. “this field should be visible only when category is “blabla”).
      7. field could be added also to attachments – but dont forget, that it does not have only the standard post.php, but that the fields are displayed also on the attachments popup screen
      8. dont forget the field versioning (with older version of post you see also older version of its fields)
      9. posibility to create groups of the fields with their conditions
      10. possibility to e.g. display fields in two columns
      11. very advanced field – repeater field (e.g. something like this http://www.advancedcustomfields.com/add-ons/repeater-field/) – possibility to create field table. This is very complex, but very often this is the only way how some things can be solved, as currently there is only simple ID->key->value in WP, but very often you need ID->key->array(subkeyN->valueN), so without this field type many users will still have to use some solution as advanced custom fields.

      • Scott Kingsley Clark 5:26 pm on August 4, 2015 Permalink | Log in to Reply

        1. That sounds great, it sounds like Fields API could enable this through the options you pass into the configurations. A plugin could pick that up and change how the fields work from there, per language.

        2. That’s on the list at the moment

        3. Good ideas, we’re still nailing down the specs but those are two I’d like to see us tackle if possible — it’s just going to be difficult.

        4. Not currently possible, but when Term meta makes it’s way into core, this Fields API will be there to utilize 🙂

        5. Sounds great, I’m not sure how much of this we’ll tackle in the Fields API itself, but it’ll definitely be doable by a plugin.

        6. That also sounds great, plugin material though — but enabled by our extensible code.

        7. This sounds great too, it’s not part of our initial focus but there are hooks in place that we could implement this for media fields as well.

        8. This isn’t a problem, everything operates by ID, revisions get their own ID, so everything should continue to work there, even when post meta revisions makes it’s way into core.

        9. Groups of fields are possible, but conditions is currently plugin territory.

        10. That sounds like plugin territory, but totally possible because we have hooks for that.

        11. I love the idea of a loop field or some basic repeatable field in core, not sure if we’ll include the field type itself in core but we’ll do everything in our power to enable developers to make use of it in the numerous plugins that have this field type as an option.

    • camu 1:16 pm on August 4, 2015 Permalink | Log in to Reply

      Thank you for all this work. I was wondering, why not just merge Advanced Custom Fields into core? It’s a robust plugin that already does most of the things listed here above. I’ve been using it for years now, and it has always delivered.

      • Scott Kingsley Clark 4:04 pm on August 4, 2015 Permalink | Log in to Reply

        There aren’t any plugins that could be a candidate to merge into WordPress core right now as they are, this Fields API effort is our focus right now to make a better API for custom fields plugins out there to utilize for a better standard when developing for WordPress. What we’re doing here wouldn’t replace custom fields plugins for many developers, they add features that are useful for some projects and offer an admin UI to manage these things. At this point, the focus is not on creating an admin UI to manage custom fields for objects in WordPress, but to offer up a unified API to interact with fields for all object types.

        • khromov 4:53 pm on August 4, 2015 Permalink | Log in to Reply

          From reading the post, the proposed direction is very similar to what ACF offers. There will be edge cases that the new API won’t fill right away (ie. repeatable fields), but it’s something the community is going to want and build.

          My point is, don’t understimate the will of the community to bend this to their particular use case. A good design (which it looks like so far), enables that.

          • Scott Kingsley Clark 4:57 pm on August 4, 2015 Permalink | Log in to Reply

            Totally agree, this is actually the result of lots of communication between many developers of plugins / libraries for building custom fields. There’s no way I would underestimate this community, was just noting that none of the plugins were a direct fit.

    • christopherabalos 7:49 pm on August 4, 2015 Permalink | Log in to Reply

      Very interesting take on a much needed API. The WordPress API for creating metaboxes, post types, taxonomies, customizer fields, widgets, etc are in need of a uniform API. I’ve been attempting to tackle this the last few months in my own projects. My implementation revolves around new manager classes for things like post types and settings that follow a pattern similar to the Customizer API. My manager classes have been written to accept WP_Customize_Control objects.

      I’d also like to see the ability to create fields for widgets, which is something I’ve included in my implementation.

      I’m curious to hear more about the decision to wrap the fields API into a single manager vs. having separate managers that all accept the same field objects.

      Keep up the great work Scott!

      • Scott Kingsley Clark 3:21 pm on August 5, 2015 Permalink | Log in to Reply

        Widgets is totally a logical next step in this, I would love to see Fields API used for Widgets, for the Shortcode UI project, etc.

      • Scott Kingsley Clark 3:24 pm on August 5, 2015 Permalink | Log in to Reply

        One manager class is based on the Customizer API structure. I ended up going with one class so that there was one class things would be used to interact with, stored screens/sections/fields/controls in, etc.

    • Chris Lema 3:43 am on August 5, 2015 Permalink | Log in to Reply

      My question may be silly, but assuming that a field has been created and attached to a post type, and then populated for all those posts, is there an expected facility in the API to be able to return a list of posts where the field is populated with a specific value? Additionally, are their collection operations where I could get a count of all those posts with the value of a specific field matching what I’m submitting?

      • Scott Kingsley Clark 3:20 pm on August 5, 2015 Permalink | Log in to Reply

        WP_Query / meta_query already do this 🙂

        The Fields API is not meant for querying for data, it’s to register the structures of information a site needs for it’s content / data types.

        • jbmanos 11:38 pm on August 7, 2015 Permalink | Log in to Reply

          So if I’m understanding this correctly, I could register fields I’ve developed for a particular site and the core will know what fields it needs to show the data it has? In other words, this would be a manifest of information types available?

          If so, I like it! I already get a little nervous that if I lose a piece of php that registered the fields to begin with, how many SQL queries and guess it’ll take to recover the data.

          • Scott Kingsley Clark 6:50 am on August 10, 2015 Permalink | Log in to Reply

            Basically, yes. A standardized API to register fields for different objects. An API that would eventually be the under pinning of all other Form UI, in that — a plugin, theme, or app (as exposed by a REST API) could know everything about your WP site, what content types it has (already possible), down to what fields it has (already possible) but more importantly — what types of fields it has. More in-depth information about the fields, where they are, how they’re used.

            If you lose the PHP side of this, you’re back to where you were before, which is to say, the same place you’d find yourself if you lost the config in any other plugin that builds fields. So there’s not really anything to get around that 🙂

    • Paul de Wouters 6:35 am on August 5, 2015 Permalink | Log in to Reply

      Not sure if it has been mentioned, but custom fields for menu items would be useful too.

      • Scott Kingsley Clark 3:20 pm on August 5, 2015 Permalink | Log in to Reply

        That’s a whole other story for another time, but I agree. Menu item custom fields are currently a very big affair to deal with given the lack of actions/filters in that area of the UI.

    • Eric Andrew Lewis 7:24 pm on August 14, 2015 Permalink | Log in to Reply

      Defining a spec for this project is a great way to go.

      The spec can be drawn out in some places. e.g. What is a field, and how does it relate to object metadata? How does this relate to prior HTML form HTTP POST submissions? This would help describe the problem space.

      The cross-platform usage of self-describing fields/metadata for rendering custom views (e.g. rendering a color-picker in the iOS app, or rendering custom media fields in Attachment Details) is a major plus, which should be explained in Community Benefits.

      The “Background” section (as-is) doesn’t seem relevant in spec, as it describes the history of the project, not specification.

  • Dominik Schilling (ocean90) 8:38 pm on July 30, 2015 Permalink
    Tags: , , options-meta   

    get_transient() is now more strict in 4.3 

    WordPress 4.3 includes a change to get_transient(). As reported in #23881 and #30380, get_transient() had a check for get_option( $transient_timeout ) < time().

    Because get_option() can return false and false is always < time() 😖, get_transient() could delete transient timeout options in an unexpected way or cause two more unnecessary queries.

    WordPress 4.3 now checks the return value first before comparing with the current time. This means that WordPress will no longer delete broken transients via get_transient() if you have only deleted the timeout option. See [33110].

    If you have to delete a transient manually, please make sure that you’re deleting _transient_timeout_$transient and '_transient_' . $transient. (Hint: Transients can be stored in an external object cache too.)

    See also: the Transients API Codex page.

     
    • webaware 2:09 am on July 31, 2015 Permalink | Log in to Reply

      Delete Expired Transients will delete those orphan transient timeouts automatically, which might be an easier option for some.

    • 99robots 8:58 pm on August 18, 2015 Permalink | Log in to Reply

      Great Stuff 🙂

    • Steve Truman 5:39 am on September 4, 2015 Permalink | Log in to Reply

      @ocean90 – just letting you know (if you do not already) that the change to get_transient is a very big problem for Premium Plugin and Theme developers and a potential timebomb for WordPress users of Premium products.

      Our plugins like most Premium Licensed plugins and themes auto check for new versions using transient to call to the a3api once each 24 hours.

      The obscure change to the transient has deleted that. We alone now 9,000 plus licensed plugins that no longer call to the a3api (if they have upgraded to WP 4.3) and can never know that there is a new version. I know that this affects many developers premium plugins.

      This of course is a massive security issue over time.

      It is an easy enough fix by adding this code

      `// Fixed for work compatibility WP 4.3 when transient_timeout is deleted
      if ( false !== $respone_api ) {
      $transient_timeout = ‘_transient_timeout_’ . ‘wc_psad_update_info’;
      $timeout = get_option( $transient_timeout, false );
      if ( false === $timeout ) {
      $respone_api = false;
      }
      }`

      That is not the issue – because adding that in a new version is of no help as the user never sees an upgrade notice to get the code if they have already upgraded to WP 4.3

      I do not know how many WP users will be affected by this as just about every Premium Plugins has the issue now – example Gravity Forms and All WooThemes WooCommerce plugins

      There is not much point in emailing customers to tell them about it and ask them to deactivate and reactivate the plugin or theme (which will trigger a manual call to the a3api and get the new version for auto upgrade via core updates) as email open rates on mass mail outs are low.

      I have absolutely no idea what you can do about it or if you are even interested in rectifying the situation – but just be aware that there will be many 100,000’s of thousands of Premium plugins and themes sitting on WordPress sites that will never be updated again and that is a massive security issue.

      Regards
      Steve

      • Ipstenu (Mika Epstein) 2:39 pm on September 4, 2015 Permalink | Log in to Reply

        I think @mordauk may be better equipped to explain it, but can’t you hook into wp’s update checker and just add your own API to it? That was always the preferred method as I understood it, and uses WP-cron. I know EDD does that to look for updates to my add ons.

        • Pippin Williamson 2:46 pm on September 4, 2015 Permalink | Log in to Reply

          Yes, hooking into the standard update API is a much better solution, but even without using it, this change should not break update checks added to commercial plugins.

          The only time (correct me if I’m wrong) that this change will break custom update checks is if one is forcing an update check to run by deleting the transient timeout in the wp_options table.

          A transient that has naturally expired still functions exactly the same as it always has. Yes?

    • Steve Truman 6:42 am on September 5, 2015 Permalink | Log in to Reply

      Hi Mika and Pippin – thanks for your replies. We used to use the standard update API of WP but it caused issues as the API call not only checks for new version but also checks the “SERVER_ADDR” to verify the License key. This was failing due to a known and unresolved WP_CLI bug – wp doesn’t fully populate $_SERVER array https://github.com/wp-cli/wp-cli/issues/1734

      As a work around for the issue we created a fall back cron that if the “SERER_ADDR” fails then the call is scheduled for every 2 hours for 10 retries. Doing this completely resolved the issue.

      This is the code
      //Getting version number
      $respone_api = get_transient(“wc_psad_update_info”);
      if ( ! $cache ) {
      $respone_api = false;
      }
      // If transient is expired or is not existed then connect to a3api server to get new version and save to transient as cached with timeout is 24 hours or 2 hours for some way can’t connect to a3api server
      if ( ! $respone_api ) {
      //connect to a3api server to check new version
      }

      on WP 4.2.4 or older
      if ‘_transient_timeout_wc_psad_update_info’ is deleted by any reason
      $respone_api = get_transient(“wc_psad_update_info”);
      will return ‘false’
      and is correct to our plugin connect to a3api server for check new version

      With WP 4.3
      if ‘_transient_timeout_wc_psad_update_info’ is deleted by any reason
      $respone_api = get_transient(“wc_psad_update_info”);
      will not return ‘false’, it return cached value ( this cached value is never removed if ‘_transient_timeout_wc_psad_update_info’ is deleted )
      and it never go to
      if ( ! $respone_api ) {
      //connect to a3api server to check new version
      }
      to get new version

      Hence the plugin can never call for new version – user has no idea and just thinks there is no updates. We have a developers License for Gravity Forms and know it has the same issue as well as WooThemes WooCommerce plugins

      As I wrote the fix is easy – what is not easy or clear is how let License holders know that there is an updated version with the patch.

      I note the update was “get_transient() is now more strict in 4.3”

      I am not aware of any issues with the old get_transient function? Was there security or other issues that required the change?

      As I wrote we are not the only Premium plugin developers who have this issue and unless it is addressed there will be 100,000’s of wp site with plugins or themes that never show new version upgrades and hence over time will become a security issue.

      • Pippin Williamson 1:27 am on September 6, 2015 Permalink | Log in to Reply

        aWhat you’ve described makes sense as a way for update detection, but doesn’t follow what core has changed.

        The only reason your transients would no longer be working is if you (or someone else) is deleting the _timeout value without deleting the rest of the transient.

        One possibility for you.

        Don’t do: if ( ! $cache ) {

        Do: if ( false !== $cache ) {

      • Ipstenu (Mika Epstein) 4:36 am on September 6, 2015 Permalink | Log in to Reply

        The issue with the old get_transient was they weren’t deleting properly. They could “could delete transient timeout options in an unexpected way or cause two more unnecessary queries.” In the race for speed, the queries were killing some heavy sites.

        Looking at your explanation:

        > This was failing due to a known and unresolved WP_CLI bug – wp doesn’t fully populate $_SERVER array

        Without looking at the code that was failing I can only ask “Why did you chose to do this instead of not using $_SERVER?” Looking at that bug (https://github.com/wp-cli/wp-cli/issues/785 is the current one), using dirname(__FILE__) was a good fix, and FWIW, I’ve actually had $_SERVER in my wp-config and it does ‘break’ wp-cli but how does that break your code? My updates have trucked along just fine with wp-cli being weird. I don’t get how they’re related.

        Also where was the SERVER array in your code, what was it being used for, and were there other alternatives? I’d love to see if we can fix this the ‘right’ way … Though IMO the ‘wrong’ is because wp-cli is borked because PHP (yeah, it’s PHP) is stupid. I’ll drop a note to wp-cli about it.

        See http://stackoverflow.com/questions/626416/php-server-name-from-command-line

        The answer for “How do I inform my users?” is one presumes you have their contact information? So … email. It sucks but a “Hey, sorry about this but we didn’t notice a change in WP conflicted with our update checker. You need to manually update. We’re REALLY sorry. Here have a free orange.” Or something like that.

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel
Skip to toolbar