WordPress.org

Make WordPress Core

Updates from Gary Pendergast Toggle Comment Threads | Keyboard Shortcuts

  • Gary Pendergast 2:33 am on November 24, 2016 Permalink |
    Tags: ,   

    New Committers! 

    Usually, new committers are announced in line with release cycles, but we were all just too excited to wait until the 4.8 cycle started, so here they are!

    First up, James Nylen (@jnylen0). James has been a driving force on the REST API, both when it was a feature plugin, and more recently in Core since the endpoints were merged. The tickets and comments he leaves on Trac are always thorough and thoughtful, his patches are consistently excellent, and his attitude is exemplary.

    Next, Adam Silverstein (@adamsilverstein). Adam has been a regular contributor for years, bringing significant improvements to the Media Grid, as well as handling large parts of the JavaScript work around the REST API endpoints – both in wp-api.js, and tenaciously working on porting Dashboard features across to using the endpoints.

    Rounding out our guest committer list, Felix Arntz (@flixos90). Felix has been a contributor to Multisite for some time now, writing excellent patches, as well as running Office Hours and Bug Scrubs. Not only that, he’s always been willing to jump in and help in any area of Core, showing the same level of enthusiasm and consideration across the board.

    Finally, we have a bumper class of guest committers to make make permanent! Pascal Birchler (@swissspidy) and Joe McGill (@joemcgill), Rachel Baker (@rachelbaker), and Mike Schroder (@mikeschroder) are now permanent committers.

    Please join me in congratulating James, Adam, Felix, Pascal, Joe, Rachel, and Mike! 🎉🔥⭐️👻💯

     
  • Gary Pendergast 4:07 am on September 8, 2016 Permalink |
    Tags: , , , Hooks,   

    WP_Hook: Next Generation Actions and Filters 

    WordPress 4.7 introduces a significant reworking of action and filter iteration to address bugs that arose from recursive callbacks and from callbacks that changed the hooked callbacks on currently running actions/filters.

    What does this mean for you, the plugin or theme developer? In almost all cases, nothing. Everything should continue to run as expected, and this should fix a number of hard-to-trace bugs when different plugins are stepping on each others callbacks.

    Who is affected?

    If your plugin directly accesses the $wp_filter global rather than using the public hooks API, you might run into compatibility issues.

    Case 1: Directly setting callbacks in the $wp_filter array

    $wp_filter['save_post'][10]['my_special_key'] = array( 'function' => 'my_callback_function', 'accepted_args' => 2 );

    This will no longer work, and will cause a fatal error. $wp_filter['save_post'] is no longer a simple array. Rather, it is an object that implements the ArrayAccess interface.

    You have two options to work around. The first (and preferred) method is to use the add_filter() or add_action() functions.

    add_action( 'save_post', 'my_callback_function', 10, 2 );

    If, for some reason, you absolutely cannot, you can still work around this.

    if ( ! isset( $wp_filter['save_post'] ) ) {
        $wp_filter['save_post'] = new WP_Hook();
    }
    $wp_filter[ 'save_post' ]->callbacks[10]['my_special_key'] = array( 'function' => 'my_callback_function', 'accepted_args' => 2 );

    Case 2: Directly unsetting callbacks in the $wp_filter array

    unset( $wp_filter['save_post'][10][ $my_callback_id ] );

    This will fail for the same reason as case one. To work around, you can use the standard remove_filter() / remove_action() functions.

    remove_action( 'save_post', 'my_callback_function', 10, 2 );

    Or, if you absolutely must access the array directly:

    if ( isset( $wp_filter[ 'save_post' ] ) ) {
        unset( $wp_filter['save_post']->callbacks[10][ $my_callback_id ] );
    }

    Case 3: Checking if a hook is an array

    if ( isset( $wp_filter['save_post'] ) && is_array( $wp_filter['save_post'] ) ) {
        // do something
    }

    This will always return false. $wp_filter['save_post'] is a WP_Hook object. To check if a hook has callbacks, use has_action() or has_filter().

    if ( has_action( 'save_post' ) ) {
        // do something
    }

    Case 4: Moving the $wp_filter array pointer manually

    If you’re calling next() or prev() on the $wp_filter array pointer to manually manage the order that callbacks are called in (or if you’re doing it to work around #17817), you will likely be unsuccessful. Use callback priorities, add_action() / add_filter(), and remove_action() / remove_filter() appropriately and let WordPress manage execution order.

    Other Cases

    Almost all other cases where you might manipulate the $wp_filter global directly should continue to function as expected. The WP_Hook object implements the ArrayAccess and IteratorAggregate interfaces so that, while it’s not recommended, you may continue to iterate over callbacks in $wp_filter or directly retrieve priorities from the callback array.

    Props

    While there were many contributors of the course of this ticket, all of whom are listed in the changeset, I’d like to especially thank @jbrinley, for his work in architecting this solution, researching and testing potential compatibility issues, and for authoring the bulk of this post.

    What’s Next

    We’re confident in how solid the code is for the majority of sites, and for the major edge cases, so now it’s time to add the code to the nightly build, so all y’all can test it against your sites and plugins.

    This is being treated as an early commit of a feature project, so the final decision for whether this code will be kept in WordPress 4.7 will be made no later than beta 1, which gives us a month and a half to see how it effects usage in the wider world.

    The current nightly build contains this update, for your testing enjoyment.

    If you have any feedback on the code, please leave a comment on this post. Please post any bugs you find to Core Trac.

     
    • Aaron Jorbin 4:27 am on September 8, 2016 Permalink | Log in to Reply

      One additional thing to note is that if you were directly accessing the $wp_filter global in your advance-cache.php, this is no longer necessary as of 4.6. In 4.7, `ABSPATH . WPINC . ‘/plugin.php’` is included with `require_once`, so calling it before then such as in an auto-prepend file or an alternative entry path than index.php (such as what wp-cli uses), you can safely initialize the plugin API early.

      • David Anderson 9:21 am on September 8, 2016 Permalink | Log in to Reply

        I use an auto-prepend file on a shared-hosting server, where the users have a variety of WP versions. (I hook a function to muplugins_loaded, and then in there hook all the things I wanted to do – disable unused/problematic XMLRPC functions, add some SEO spammers to robots.txt, make heartbeat less aggressive – that sort of thing).

        The problem is, at the point that an auto-prepend file runs, you don’t know what version of WP is going to load… and in consequence, don’t know what’s right to put into $wp_filter. The old way had the big plus of being safe even if the .php file wasn’t even part of WordPress.

        I was about to suggest a solution… however…. now that I look at class-wp-hook.php, it looks like WP_Hook::build_preinitialized_hooks() is already doing what I was about to suggest, namely, seeing if $wp_filter has already got entries, and converting them when WP_Hook is available. So, am I right in thinking that a case like mine is already covered / should “just work” ?

        • Jonathan Brinley 12:42 pm on September 8, 2016 Permalink | Log in to Reply

          That is correct. If you set up hooks the old way before the WP_Hook class exists, then they will be converted. This was initially done as a workaround when advanced-cache.php (and also automated tests) loaded before `add_action` was available. But it also solves your case.

        • Aaron Jorbin 3:32 pm on September 8, 2016 Permalink | Log in to Reply

          I would encourage you to keep all the sites you host updated to the latest supported version of WordPress. This will prevent you from having any issue requiring plugin.php once 4.7 is released.

    • Thorsten Frommen 6:15 am on September 8, 2016 Permalink | Log in to Reply

      Why again is the class final without implementing a hook-specific interface?

      • Gary Pendergast 6:53 am on September 8, 2016 Permalink | Log in to Reply

        WP_Hook is a fundamental part of how WordPress works, so making it extendable in any way is something that we’d have to consider very seriously, particularly around maintaining backwards compatibility.

        Do you have a particular use case for such an interface? I’m not against either removing the final or adding an interface, but I would like to know what you had in mind.

        • toscho 8:00 am on September 8, 2016 Permalink | Log in to Reply

          Type hinting with the ability to use a mock or spy would be very useful. Just *never* declare something as `final` that doesn’t implement an interface. That’s always a bug in public code.

        • Thorsten Frommen 5:58 am on September 9, 2016 Permalink | Log in to Reply

          TL;DR: What toscho said. 🙂

          I didn’t think of a concrete use case here, but more in a general sense. The first thing that comes to mind, though, is indeed testing (or more specifically: mocking).

          Another thing, which I think might be true for several comments on several tickets/posts: just by making something important accessible doesn’t mean the world will eventually end because of that change. Yes, the concept of actions and filters is a fundamental part of WordPress’s internal behavior. Allowing someone to replace that behavior isn’t a bad thing, though. I mean, currently, any bad plugin can just do unset( $GLOBALS['wp_filter'] );. That has been, still is and always will be the case. If someone has access to some machinery they can do pretty much with that. If you don’t want this to happen, then you might have to prevent people gaining access. Something like $GLOBALS['wpdb']->query( 'DROP DATABASE;' ); isn’t anything you can avoid when someone has both access to the DBAL and authorization to perform such a query.

          Does this make sense?

        • screamingdev 9:43 pm on September 18, 2016 Permalink | Log in to Reply

          If you remove the `final` that would be great.
          I know WP_Post and some others are final too but once Nacin promised that the final statements of those will go away as soon as every post type has his own class.

          I saw a few commits and how they clean things up. Leave that `final` and every developer will thank you for that. In my case it is rather OOP / extending than testing. So, yes, an interface is always great if the class is injected somewhere.

          Hope to see no more final anywhere ^^ Cheers!

    • jadpm 7:37 am on September 8, 2016 Permalink | Log in to Reply

      Nice. This is going to help us remove some hacky code, or at least deactivate it from 4.7 on. Thanks a lot.

    • Amir Helzer 8:06 am on September 8, 2016 Permalink | Log in to Reply

      Folks, thanks a lot for announcing this in advance. This helps us (and others) get prepared on time. Highly appreciated!

    • Alex Mills (Viper007Bond) 9:23 am on September 8, 2016 Permalink | Log in to Reply

      I just wanted to compliment everyone who worked on this. What a monumental task this was. Great on you guys for having the perseverance to see it though! I ran into this bug countless times without realizing it and it’ll be very nice to have it resolved.

    • eddr 9:50 am on September 8, 2016 Permalink | Log in to Reply

      Great!
      Would be nice to have a good dociumentation of all the filters and actions, and/or an ordered list

    • Ahmad Awais 11:14 pm on September 8, 2016 Permalink | Log in to Reply

      Noice! Have reported it back to a few plugins.

      • Gary Pendergast 12:11 am on September 9, 2016 Permalink | Log in to Reply

        Have you noticed plugins that are not working correctly after this changeset? Or is a pro-active review of code that touches $wp_filter?

        If there are plugins that are not working, please send a list of them through – I’d like to make sure as many edge cases are fixed in core as possible.

    • Giuseppe Mazzapica 8:19 am on September 9, 2016 Permalink | Log in to Reply

      The only place where I touch the global directly is in some edge cases where I need to add action and filters _before_ WordPress has been loaded. I had no time yet to test against trunk, but looking at class code it seems that any content of $wp_filter is converted to new format when WP is loaded. Is that right? Will any code that fills $wp_filter before bootstrapping WordPress fail after this change?

      • Dion Hulse 5:18 am on September 10, 2016 Permalink | Log in to Reply

        That’s correct; Filters added to the globals BEFORE WordPress is loaded still works, it’s just upgraded on the fly.
        If you have a custom script, as explained in the comments here, you can also include `wp-includes/plugin.php` early to get access to the API properly.

    • Josh Levinson 1:51 am on September 14, 2016 Permalink | Log in to Reply

      To avoid confusion regarding `WP_Hook::build_preinitialized_hooks`, it may be worth better defining the scope in which it’s useful/can provide some degree of backwards compatibility.

      *`build_preinitialized_hooks` only “upgrades” hooks that are added *prior to the plugin API being loaded.*

      If any hooks are added in the style of the example given in Case 1 by OP _after_ the plugin API is loaded, _fatals will occur_. By then, `build_preinitialized_hooks` has already upgraded any filters that came before it.

      See https://github.com/Automattic/batcache/pull/73 for an example of this in the wild.

    • Matt Mullenweg 12:59 pm on November 2, 2016 Permalink | Log in to Reply

      How does it benchmark before and after?

      • Gary Pendergast 11:48 pm on November 2, 2016 Permalink | Log in to Reply

        Memory: The new method uses ~3% more memory than the old method (just loading the relevant code, not all of WordPress), which translates to around 10-20KB.

        For time, the new method is ~10% faster when a hook has 0 callbacks, 5-15% slower with 1-3 callbacks, then gets progressively faster as more callbacks are added – 25% with 100 callbacks, for example.

        This translates to a slight (1-2%, close enough to not really be statistically significant) performance improvement in actual page loads – on the front page of my test WordPress install with Jetpack and Twenty Seventeen activated, there are 3652 calls to do_action() and apply_filters(), 3301 of which have 0 callbacks, 327 have 1-2 callbacks, and 24 have 4+.

    • leemon 3:08 pm on November 25, 2016 Permalink | Log in to Reply

      I have a couple of plugins with calls such as: `add_action( ‘wp_enqueue_scripts’, array( ‘NameOfClass’, ‘my_enqueue_scripts’ ) )`. Since upgrading to 4.7RC I’m getting a “Deprecated: Non-static method NameOfClass::my_enqueue_scripts() should not be called statically” warning. Is there an alternative to using `array($this, ‘my_enqueue_scripts’ )`?
      Thanks

      • Gary Pendergast 4:44 am on November 26, 2016 Permalink | Log in to Reply

        Thank you for the bug report! That deprecation notice is coming from PHP 5.6 – can you confirm that you see the notice running WordPress 4.6 under PHP 5.6? If you don’t see if, please open a ticket on our bug tracker, so we can investigate it in more detail.

        • leemon 5:06 am on November 26, 2016 Permalink | Log in to Reply

          Yes, I’m seeing this running 4.6, too. Sorry. I have no idea why I didn’t catch it earlier.

          • Gary Pendergast 11:32 am on November 26, 2016 Permalink | Log in to Reply

            I’ve just been trying to test this, but I haven’t been able to reproduce it – I get the same notice under WordPress 4.6 as I do under WordPress 4.7.

            Could you please open a ticket on https://core.trac.wordpress.org, and attach a test case to reproduce it?

            • leemon 11:40 pm on November 26, 2016 Permalink

              Yes, that’s what I was trying to say, sorry. I’m seeing the same notice under 4.6 and 4.7, so this is the expected behavior and that means that I must change how I add hooks in my plugins. Thanks!

    • Paul Bearne 1:31 pm on November 28, 2016 Permalink | Log in to Reply

      Just reading the post I think some code that I look after is OK but just wanted to check

      “`
      // Remove all the_content filters so child posts are not filtered
      // removing share, vote and other per-post items from the live update stream.
      // Store the filters first for later restoration so filters still fire outside the update stream
      $stored_wp_filter_the_content = $wp_filter;
      $this->clear_most_the_content_filters();

      do some stuff

      // Restore the_content filters and carry on
      $wp_filter = $stored_wp_filter_the_content;
      “`

      and in the clear_most_the_content_filters();

      “`

      /**
      * Adjust the_content filter removing all but a handful of whitelisted filters,
      * preventing plugins from adding content
      *
      * @since 1.0.9
      */
      private function clear_most_the_content_filters() {
      global $wp_filter;

      // do we have the_content and is not empty
      if ( isset( $wp_filter[‘the_content’] ) && empty( $wp_filter[‘the_content’] ) ) {

      return;
      }

      // is it an WP_Hook class
      if ( ! is_object( $wp_filter[‘the_content’] ) || ‘WP_Hook’ !== get_class( $wp_filter[‘the_content’] ) || ! isset( $wp_filter[‘the_content’]->callbacks ) ) {

      return;
      }

      /**
      * Filter to allow to override the White list of the filters we want to preserve.
      *
      * @since 1.3.5.2
      *
      * @param array $whitelisted_content_filters White list of the filters we want to preserve.
      *
      */
      $whitelisted_content_filters = apply_filters( ‘whitelisted_content_filters’,
      array(
      ‘process_oembeds’,
      ‘run_shortcode’,
      ‘autoembed’,
      ‘wptexturize’,
      ‘convert_smilies’,
      ‘convert_chars’,
      ‘wpautop’,
      ‘shortcode_unautop’,
      ‘capital_P_dangit’,
      ‘do_shortcode’,
      ‘add_children_to_post’,
      )
      );

      // Iterate thru all existing the_content filters
      foreach ( $wp_filter[‘the_content’] as $filter_key => $filter_value ) {
      // Filters are in arrays by priority, so iterate thru each of those
      foreach ( $filter_value as $content_filter_key => $content_filter_value ) {
      $found_in_whitelist = false;
      // Loop thru the whitelisted filters to see if this filter should be unset
      foreach ( $whitelisted_content_filters as $white ) {
      if ( false !== strpos( $content_filter_key, $white ) ) {
      $found_in_whitelist = true;
      break;
      }
      }
      if ( ! $found_in_whitelist ) {
      remove_filter( ‘the_content’, $content_filter_key, $filter_key );
      }
      }
      }

      return;
      }

      “`

      any thoughts

  • Gary Pendergast 4:31 am on March 7, 2016 Permalink
    Tags: , reactions   

    Reactions 

    What are Reactions?

    If you’ve used Slack or Thefacebook recently, you’ll have noticed a new way of interacting and providing feedback – emoji reactions. It works much the same way as a Like button, but provides a wider range of reactions, so readers can give more nuanced feedback, without needing to go to the effort of leaving a comment. This also allows for readers to provide the same level of interaction in situations where a “Like” is an inappropriate message to send, as Eric Meyer describes in his post about Inadvertent Algorithmic Cruelty.

    What does it do?

    The reactions plugin currently has the following features:

    • Allows for reactions to posts
    • REST API endpoints for storing and retrieving reactions
    • An exceedingly ugly emoji selector

    What is still being worked on?

    Pretty much everything!

    In its current state, the plugin is mostly a proof of concept, in need of significant work improving edge cases, design and User Experience.

    Your first step is to install the plugin, as well as the WP-API plugin (the WP-API plugin is currently a requirement to avoid code duplication, that will likely be re-evaluated based on when each plugin might be merged into Core).

    Please report any issues on the Github repository, or drop in the #feature-reactions channel in Slack to ask questions or give feedback. It’s also where we have our weekly chats, on Wednesday 23:00 UTC. Thank you!

     
    • Jeremy Felt 5:04 am on March 7, 2016 Permalink | Log in to Reply

      💯💯💯

    • jeffmcneill 5:16 am on March 7, 2016 Permalink | Log in to Reply

      It would be helpful, besides extending comments or adding “reaction” metadata to pages/posts, to enable for smilies and/or emoji. The limited range of reactions, especially in the age of the chat sticker/emoji sets, seems already out of date.

      • Gary Pendergast 5:55 am on March 7, 2016 Permalink | Log in to Reply

        This would be an interesting direction to take it – I’m not sure that there’s a good way to make a usable, extendible system, but it’s definitely worth exploring. 🙂

    • Mark-k 5:46 am on March 7, 2016 Permalink | Log in to Reply

      WTF is this stupid horrible idea? is everybody on the internet has the expression skill of a 5 year old child that we need to “help” them express themselves? Did wordpress change its motto from “democrizing web publishing” to “dumbing down the web”

      [edited for language]

      • Helen Hou-Sandi 3:21 am on March 8, 2016 Permalink | Log in to Reply

        I have untrashed this comment. It is extraordinarily rude (even more so before I edited it) and I am almost impressed at how non-constructive it is, but so long as you continue to make such comments I think the ones that stay away from personal attacks might as well stand on display for others to judge as they might, especially in aggregate. I will not be restoring the comments from yourself and another party that are both rather irrelevant to the actual topic at hand and personal in nature.

        As for your question “is everybody on the internet has the expression skill of a 5 year old child that we need to ‘help’ them express themselves?”, your comment itself answers that question quite well. A simple thumbs down would have sufficed, I think.

        • Mark-k 12:59 pm on March 8, 2016 Permalink | Log in to Reply

          The other part which is censored now, was also not rude at all. I tend to not be rude towards people I don’t know. If it is not constructive to say that wordpress site owners do not need yet another bloated feature built on the bloated an incomprehensible emoji, then I am not sure what kind of constructive discussion is expected here.

          OK I will mind my language from now on, or even better, just avoid commenting since it is apparent there is some hidden code of conduct which is on a “need to know” base, and I do not rank high enough to know it, but you can’t say that on Otto, and his comment is still trashed.

          There was another comment that was deleted and the only commonality between them is that they were against the idea of such a thing getting to the core, 3 different people using 3 different ways to express themselves. What I get from this episode is that wordpress core developers use the posts on “make” just as a lip service to community involvement, but true community involvement not really desired.

    • modemlooper 5:53 am on March 7, 2016 Permalink | Log in to Reply

      I have to agree with the comment that was deleted, this seems like something for a child. Why is this even a consideration?

      • Gary Pendergast 5:59 am on March 7, 2016 Permalink | Log in to Reply

        Right now, it isn’t being considered for Core – it’s being explored as possible feature in the future. The idea still has to prove itself in terms of usefulness, usability, and general appeal. In terms of how close this is to landing in Core, it’s about the same as a new ticket being opened on Trac.

        Regarding the deleted comment, everyone is welcome to share their opinion, positive or negative. Aggressive and offensive comments will not be tolerated under any circumstances, however.

        • Mike 7:40 am on March 7, 2016 Permalink | Log in to Reply

          Why are comments being deleted? If you must, please invite the “offender” an opportunity to rephrase.

          • Gary Pendergast 7:51 am on March 7, 2016 Permalink | Log in to Reply

            Nobody has been banned, everyone is welcome to rephrase their feedback and resubmit.

            Just as I wouldn’t walk into someone’s workplace and abuse them, i expect the same courtesy in my (virtual) workplace.

            • Mike 10:04 am on March 8, 2016 Permalink

              Did I say banned? No.
              BTW this is the WordPress.org workplace.
              And as Helen pointed out he didn’t engage in a personal attack.
              Anyway I don’t have an opinion but I am better informed now.
              Bring back frameworks!

    • Hans-Helge Buerger 8:44 am on March 7, 2016 Permalink | Log in to Reply

      Great idea. I recently found this https://emoji.rodeo/ and though about putting this into a plugin. Maybe this might be an inspiration for you and a good UI.

      I also would like to see that for comments. This would be very useful to mark good or bad comments so users can easily identify them.

      • Gary Pendergast 8:47 am on March 7, 2016 Permalink | Log in to Reply

        emoji.rodeo is super cool (and an amazing domain name!) – thanks for the link, I’ll check it out!

        Comment reactions is definitely on the list, good to hear that you’re interested in it!

    • petermolnar 9:49 am on March 7, 2016 Permalink | Log in to Reply

      Some of us implemented “reacji” as regular comments with a single emoji as content, using this library:
      https://github.com/dissolve/single-emoji-recognizer

      You might find it useful for emoji detection in, for example, pingbacks.

    • andreasnrb 10:04 am on March 7, 2016 Permalink | Log in to Reply

      This sounds like a good case for a stand alone plugin. And about seeing this as a equivalent of a new track ticket. Please, really? Id say publishing this on make marks its status as way beyond a mere new trac ticket.

      • Gary Pendergast 10:08 am on March 7, 2016 Permalink | Log in to Reply

        This sounds like a good case for a stand alone plugin.

        That’s absolutely a potential outcome – one of the original ideas of feature plugins is that they may never be merged into Core – we just end up with a high quality plugin that folks can use. I’m quite okay with that happening, but we won’t know until we investigate further. 🙂

      • Utkarsh 11:11 am on March 7, 2016 Permalink | Log in to Reply

        I agree with stand alone plugin.

    • Tammie 12:16 pm on March 7, 2016 Permalink | Log in to Reply

      I really love this idea simply because it allows people to be more human. It won’t be something everyone will want, but that’s not the point. Any chance we get to encourage and allow more natural interactions in plugin or core, we should.

      There are a lot of reason emoji reactions work for humans and are used so easily and frequently. There’s some really interesting psychological theory behind it. Yes writing is human but emojis often work because a picture does say 1000 words. We connect with emojis.

      I have no opinion on if this is a plugin or should be in core. I’m just grateful and delighted it exists 🙂

    • Slava UA 12:39 pm on March 7, 2016 Permalink | Log in to Reply

      Agree with @modemlooper. Don’t see any reasons to have this in core. IMO.

    • George Stephanis 3:16 pm on March 7, 2016 Permalink | Log in to Reply

      I like this being aimed for core, if for nothing other than its inadvertently fleshing out the ability to build ‘Custom Comment Types’ to match the internal paradigms for custom post types and taxonomies.

    • George Stephanis 3:19 pm on March 7, 2016 Permalink | Log in to Reply

      Also, if we are going to use an emoji picker, this looks like a solid option —

      https://github.com/needim/wdt-emoji-bundle

    • chatmandesign 3:20 pm on March 7, 2016 Permalink | Log in to Reply

      I have to agree with what rapidly seems to be becoming the general consensus: Great idea for a plugin – if I ever setup my own personal blog, I might even use it – but I can’t imagine why this would be considered for Core. I would end up having to disable it on nearly every website I build, which are primarily business websites where this sort of goofy element would simply be inappropriate.

      • Abolfazl Ahani 2:54 pm on March 9, 2016 Permalink | Log in to Reply

        I agree with stand alone plugin.
        I would end up having to disable it on nearly every website I build, which are primarily business websites where this sort of goofy element would simply be inappropriate.

    • kimstuart 4:23 pm on March 7, 2016 Permalink | Log in to Reply

      I would also say this is not core material, but would probably get some traction as a stand alone plugin. We really should be focused on keeping core as compact and to the point as possible these days, and if we’re meant to be competing with Wix, Squarespace, Weebly, etc then the real focus should be on providing small business solutions.

    • Aaron Jorbin 6:01 pm on March 7, 2016 Permalink | Log in to Reply

      This seems like it could have amazing potential for core (and if it doesn’t make it, will at least be a great plugin). It’s innovation like this that shows why WordPress has the highest Net Promotor Score amongst small business owners.

    • Darren Ethier (nerrad) 7:19 pm on March 7, 2016 Permalink | Log in to Reply

      Love the idea! One way to address concerns of people having this in core, is to make it so themes register support for it. `add_theme_support( ‘wp_emojii_reactions’ )`

    • Pascal Birchler 9:37 pm on March 7, 2016 Permalink | Log in to Reply

      I’ve often been reading about emoji reactions in WordPress. Great to hear there’s something going on! Some screenshots would be nice though.

    • Alex Mills (Viper007Bond) 11:04 pm on March 7, 2016 Permalink | Log in to Reply

      Awesome! While this doesn’t seem like core material to me, a canonical and well maintained plugin for handling stuff like this would be great for the community at large.

      It’s always bummed me out that we don’t have more (any?) core plugins.

    • Primoz Cigler 8:15 am on March 8, 2016 Permalink | Log in to Reply

      I wish this never makes to core. Just another plugin, ok, I don’t care. But WHY is this discussed here? So if I make a new plugin can I now post a release notes here and spam everyone who’s subscribed to core newsletters?

    • Morten Rand-Hendriksen 1:28 am on March 9, 2016 Permalink | Log in to Reply

      Super late to the game, so I’ll just say these things:

      • This is plugin territory, useful for a subset of users, but not something I would think could pass an 80/20 test
      • If it is included in core, it should be easy to enable/disable both programmatically and through Admin/Customizer
      • In the spirit of democratization, accessibility, and internationalization, any character/emotion/emjoi set offered must be extensive, inclusive, and extendable, and provide sufficient accessibility features to ensure proper communication even when emotions are not displayed

      My concern (though not a blocker) with the introduction of emoji-style reactions is they are highly contextual and cultural. This is poorly reflected in current emoji implementations and causes some significant communication gaps, misunderstandings, and failures when messages cross cultures. A superficial primer: http://www.wired.com/2015/05/using-emoji-wrong/

      There is significant research being done on communication through emoji, and the preliminary results indicate these symbols create more confusion than anyone anticipated due to emotional, linguistic, national, and cultural differences between the person posting the icon and the person perceiving the icon.

      It’s complicated. I’m conflicted. I’d love to see this evolve into something useful, and I’m curious to see how people would end up using it.

    • Ryan McCue 1:50 am on March 9, 2016 Permalink | Log in to Reply

      FWIW, I love emoji reactions (and have my own plugin for them), but I don’t think they’re core material. I think it’d make a great plugin, or work well as part of Jetpack/WP.com’s Likes feature, but I don’t see them working as part of WP core itself.

    • lenwbrown 6:43 am on March 9, 2016 Permalink | Log in to Reply

      Personally, I like reactions and frequently use them on FB. Coming from old-school text-only chat, I recall many misunderstandings and plenty of frustration when it came to trying to interpret the sender’s intent in a text-only world. Reactions, like general-use emoticons, does help convey a swift and reactive (hence reaction) way of relaying a feeling or non-written observation.

      That said, this is something I feel is exclusively the territory of an optional plugin and not a core feature. As a plugin, I’d certainly be happy to have it. Even better, if you MUST include it, then replace the dang “Hello Dolly” useless plugin millions have already been very vocal about hating so much with this one. At least this way more than one in a million people would actually use the ‘included’ plugin. 🙂

    • DesignWall 9:58 am on March 9, 2016 Permalink | Log in to Reply

      It seems this feature should be developed as a plugin.

    • Helen Hou-Sandi 12:17 am on March 10, 2016 Permalink | Log in to Reply

      Projects like this are extremely valuable in what they expose (and hopefully therefore push to improve) in underlying APIs and related UI. In this case, there are a number of interesting things around custom comment types along with the general UX and various UIs around comment management that will need some pretty significant work.

      I am not yet of the thought that emoji reactions themselves are a core thing. They are very valuable in real-time conversations (e.g. Slack) where they can reduce noise, and can be used in interesting ways like for polls (including this post!), but I’d have to be convinced that this would see widespread and likely fairly consistent usage. To say nothing of the actual implementation itself. 🙂

      Furthermore, I don’t really understand why this was posted on Make/Core at all. I recognize that proposing a feature plugin is still a little bit of a nebulous process (which I am working on), especially when there isn’t a feature plugins response post to reply to or a meeting scheduled. However, given that posting here is not open to all and carries a certain weight, it seems rather inappropriate and very much unfair to have used this as a venue. I have encouraged other people with posting privileges to push proposed posts off onto their personal blogs a number of times when asked for feedback – I would have done the same here.

      • Joachim Jensen - Intox Studio 11:59 pm on March 10, 2016 Permalink | Log in to Reply

        I agree with all of this.

        This project looks interesting to me; not because of what it does, but because of how it does it.

        While I also think it shouldn’t be in Core, it seems like a nice, fresh way to:
        1) show new users how simple and easy it is to extend and customize WordPress
        2) give developers a glance of the new powerful APIs in WordPress

        Perhaps it could be shipped with WordPress instead of the “Hello Dolly” plugin.

    • leemon 8:12 am on March 10, 2016 Permalink | Log in to Reply

      This is plugin territory

    • carlhancock 2:59 pm on March 10, 2016 Permalink | Log in to Reply

      I like the concept. I actually like Facebook’s implementation of the emoji reactions. It’s clean, simple and easy to use. I also like that it limits me to only selecting the emoji reactions Facebook has implemented. In fact, iI’d even use something like this on a site I plan on building as it would be perfect for what i’d like to do.

      However, it’s definitely plugin territory for a variety of reasons. A big one of these being the whole decisions not options ethos. I like Facebook’s implementation because it actually limits you to a subset of emoji reactions.

      But even though I don’t think it’s ideal for core, I think putting the infrastructure in place in core (ie. custom comment types) to support it would be a good idea.

      If I were to implement something like this on a WordPress site I would want to do what Facebook does and limit the subset of emoji that are available to the user for consistency rather than letting them choose from the laundry list of emoji’s. This way I can then use this data to list only posts people have “liked” with a specific emoji, etc.

      Because of this and how I imagine others would use it this means i’d expect any WordPress solution for adding emoji reaction functionality would also have a UI for me to select only the emoji that I want to make available for the users to choose from. Not a hook or filter, but a UI in the admin.

      Which brings us to the issue of why this isn’t ideal for core…

      The first being the “decisions, not options” ethos. It would be unlikely that an emoji settings UI would be added to the WordPress Dashboard along with this as a core feature. I’m guessing it would be all or nothing and be left to hooks to customize it’s behavior. IMO this is something that demands an admin UI and I don’t see that happening in core.

      This combined with the 80/20 rule as it relates to adding features to core means as they would say on Shark Tank…

      “I’m out. But I love the idea, keep up the great work and i’m sure it’ll make a great plugin. You’ll already have a customer in me!” – Mark Cuban

    • Sergio Santos 3:12 pm on March 10, 2016 Permalink | Log in to Reply

      Agree with @rmccue. This should be part of Jetpack, not in the core.

    • Justin Tadlock 3:44 pm on March 10, 2016 Permalink | Log in to Reply

      While I could never see something like this being in core itself, it might be worthwhile to explore this as a core plugin (whatever happened to those?).

      Ultimately, I think the most important things in this plugin deal with custom comment types. Now that post types, taxonomies, term meta, and other really nice dev tools have been added to core, it’d be nice for us to flesh out comment types in WP. Developing an API without something fun to do with it is not much fun at all. This plugin would probably make the perfect test case for custom comment types. I’d love to see that happening with the support of the core leads.

    • wturrell 6:02 pm on March 11, 2016 Permalink | Log in to Reply

      Plugin, not core – given every other commenting feature is built as a separate plugin (including akismet, even though it’s distributed by default.)

      Incidentally, I took the time to install it and see what how it worked. Doesn’t do anything for me (literally – none of the buttons are added), so readme file will need editing if further steps are needed. WP 4.4.2, twentysixteen, PHP 7.0.4, commenting enabled, wp-api installed.

    • Dwain Maralack 11:24 pm on May 3, 2016 Permalink | Log in to Reply

      This is a good plugin idea, but not for core inclusion.

  • Gary Pendergast 12:38 pm on October 7, 2015 Permalink
    Tags: ,   

    🎉 One more committer for 4.4! 

    Please join me in welcoming a new guest committer for WordPress 4.4 — Ryan McCue (@rmccue)!

    Ryan has been contributing to the WordPress world for many years, through various patches, as well as being one of the maintainers of the SimplePie RSS library that WordPress uses.

    More recently, he started the WordPress REST API feature plugin, and has been leading the development of it for the past two years, nine months and five days (not that anyone is counting!). As the REST API comes closer to being ready for merge, Ryan having commit access is a natural progression: he can bring his expertise in the REST API directly across to WordPress Core.

    Congratulations, Ryan! 🎆💯⭐️⭐️⭐️⭐️⭐️

     
  • Gary Pendergast 12:45 am on September 23, 2015 Permalink
    Tags: , ,   

    Committer Reviews of the REST API 

    With the proposed merge of the REST API base code into Core, I’d like to try something a little bit different. The REST API is potentially going to cause one of the biggest shifts in workflow that WordPress has seen, so it’s important that all committers know how it works, and how it affects the parts of Core that they focus on.

    And so, here’s the plan. Before the REST API is merged into Core, it needs a code review from all active committers. The code being proposed for merge has been separated out into its own repo, for your viewing convenience.

    There are five areas I’d like to see covered:

    • Docs: Are the inline docs up to standard? What needs to be done before they’re ready? Official tutorials will be helpful, can they be fit into Devhub?
    • Security: Is it secure?
    • Performance: Are there any obvious performance issues in the base code? Are we encouraging plugin developers to write performant custom end points?
    • Ease of Use: Is it easy to write custom endpoints? Do we encourage quality code?
    • Forwards Compatibility: This is a little more nebulous, but can you envisage scenarios where we might need to break backwards compatibility in the future?

    Choose one or more of these focusses for your review, and tackle it from that perspective.

    You can also have a look at the proposed endpoints in the main repo, (scheduled for a later WordPress version), for inspiration on how it may interact with the areas of WordPress Core that you work on.

    Post your review as a comment here, and link to any relevant bug reports or pull requests you submit to the plugin repo.

    And finally, please have a think about how this process worked for you. I’d like this to be a model for future feature plugin merges, particularly those that touch many different areas of WordPress Core.

    PS: I’m not kidding about all active committers. I won’t hesitate to publicly shame you for holding up the REST API merge. 🙂

     
    • Ben Tremblay 2:37 am on September 23, 2015 Permalink | Log in to Reply

      For completeness?
      http://v2.wp-api.org/

    • Boone Gorges 2:57 am on September 23, 2015 Permalink | Log in to Reply

      Just spent a while reading through the api-core repository, and cross-referencing against some of the existing endpoints. This is my first time looking at the internals of the API; my previous experience is limited to writing a couple of custom endpoints for v1.

      Overall (and unsurprisingly) it seems like things have been nicely thought out. The syntax and conventions for registering endpoints seem easy to grok (they’re very WordPressy), without sacrificing flexibility and power.

      A few notes I jotted down while reading. I’ve opted not to open issues or PRs, as some of these are more questions than anything else – both for the core team and for the API team.

      • Why are we enabling JSONP by default? Is this for compatibility with browsers that don’t support CORS?
      • Is CSRF protection left to the endpoints? What will this look like without an HTML page to transport nonces?
      • I’d argue we should consider shipping WP_Rest_Controller as part of the “infrastructure” round. It’s hard to imagine building a very useful endpoint without it.
      • The API sends custom headers for deprecated functions and arguments. Should there be a corresponding header for _doing_it_wrong() notices?
      • Your first stop for fetching request data is $HTTP_RAW_POST_DATA, with a fallback to php://input. It should probably be the other way around – the global is deprecated in PHP 5.6 and removed in PHP 7.0.
      • Should error messages in the json_last_error_msg() shim be localizable?
      • Drew will probably have more to say about this, but docs formatting is inconsistent throughout, and will need a once- or twice-over.

      Thanks to the API team for their work so far!

      • Ryan McCue 6:44 am on September 23, 2015 Permalink | Log in to Reply

        Thanks for the review @boonebgorges. 🙂

        Why are we enabling JSONP by default? Is this for compatibility with browsers that don’t support CORS?

        Correct, and compatibility with servers that don’t allow sending CORS headers, or old client libraries, or etc.

        Is CSRF protection left to the endpoints? What will this look like without an HTML page to transport nonces?

        Authentication and authorisation is handled in the infrastructure. CSRF only applies to browser-based applications using cookie authentication, and you need the wp_rest nonce for that. (This is automatically output as WP_API_Settings.nonce and handled automatically when you use the JS client.)

        I’d argue we should consider shipping WP_Rest_Controller as part of the “infrastructure” round.

        The base controller is half way between infrastructure and endpoints, but it’s possible we’ll continue changing things in it, and it’s not as stable as the infrastructure proper. Realistically, it’s just a set of best practices, so it’s definitely possible to build useful endpoints without it; I have a tonne of basic endpoints written that just use straight up functions or closures, which you don’t need a controller class for.

        The API sends custom headers for deprecated functions and arguments. Should there be a corresponding header for _doing_it_wrong() notices?

        Potentially. The deprecated stuff can be trigger from request data, whereas _doing_it_wrong() is likely only triggered by plugin code, which is why we don’t handle it currently. I’d not be against adding it though.

        Your first stop for fetching request data is $HTTP_RAW_POST_DATA, with a fallback to php://input. It should probably be the other way around – the global is deprecated in PHP 5.6 and removed in PHP 7.0.

        php://input suffers from a fatal flaw: it can only be read from once. In case other code wants to use this, or we need to reuse the data ourselves, we load it into the variable ourselves. I don’t think we can change this, alas.

        Should error messages in the json_last_error_msg() shim be localizable?

        I’d argue no, for parity with the PHP built-in/extension.

        docs formatting is inconsistent throughout

        Definitely, we need a bunch of work on this.

        Thanks again!

    • Drew Jaynes 8:15 pm on September 23, 2015 Permalink | Log in to Reply

      For posterity: I’ve opened a PR for a core docs standards audit here: https://github.com/WP-API/WP-API/pull/1590

    • Aaron Jorbin 8:13 pm on September 30, 2015 Permalink | Log in to Reply

      I haven’t finished a full review, but here are some initial things:

      WP_HTTP_ResponseInterface

      Does there really need to be an WP_HTTP_ResponseInterface interface? It feels like over abstraction for the sake of abstraction.

      — WP_REST_Server —

      Many of the constants in WP_REST_Server don’t seem to serve a purpose without endpoints. I would like to see some more inline documentation around how they are expected to be used. For example, why is ACCEPT_JSON set as 128?

      serve_request feels like a very long method compared to many of the others. It seems to handle two things, serving the request, but before that, routing the request. Could this become two different methods? One that does the setting/routing and the other that does the serving?

      A couple of other small changes I would look at:

      • Add a set_default_headers method that could be called and encompass lines 248 -255. There could also be a filter on this list to easily enable other default headers being added to every request.
      • rest_ensure_response is called on the $result on both lines 321 and 337 (in the rest_post_dispatch filter). The only change between those is that if it is an error, it goes through error_to_response which returns WP_REST_Response which doesn’t need to be ensured is a response.

      Should envelope_response be public? It seems like it only is needed in once one place and sdoesn’t need to be used outside of it.

      More to come

      • Jeremy Felt 4:44 am on October 2, 2015 Permalink | Log in to Reply

        rest_ensure_response is called on the $result on both lines 321 and 337 (in the rest_post_dispatch filter). The only change between those is that if it is an error, it goes through error_to_response which returns WP_REST_Response which doesn’t need to be ensured is a response.

        This caught me up as well when reading through `WP_Rest_Server`. I’m not sure if there’s a better way to handle it, but it is tricky when reading.

      • Rachel Baker 1:01 pm on October 2, 2015 Permalink | Log in to Reply

      • Ryan McCue 6:40 am on October 7, 2015 Permalink | Log in to Reply

        Many of the constants in WP_REST_Server don’t seem to serve a purpose without endpoints.

        These constants were leftovers from way back when we had bitwise flags. I’ve killed those now. The only constants left now are RESTful verbs (`READABLE`, `CREATABLE`, `EDITABLE`, `DELETABLE` and `ALLMETHODS`).

        Should envelope_response be public? It seems like it only is needed in once one place and sdoesn’t need to be used outside of it.

        Enveloping is super useful in third-party code, and there’s no real reason to make it protected. I’d like to keep it as part of the class’ public API. 🙂

        Working on the rest of the feedback.

    • Drew Jaynes 8:15 pm on September 30, 2015 Permalink | Log in to Reply

      I noted this to Rachel separately, but good to have it mentioned here:

      The wp-api.js file should be renamed to wp-rest-api.js, and the handle changed to ‘wp-rest-api’ or ‘rest-api’. In the broader scheme of WordPress, this is another of many APIs, not the WordPress API any longer 🙂

    • Dominik Schilling (ocean90) 8:16 pm on September 30, 2015 Permalink | Log in to Reply

      WP_HTTP_ResponseInterface – Do we need it? If it gets released once there is no way to extend it without potentially breaking plugins which are using this class.

      • Rachel Baker 12:59 pm on October 2, 2015 Permalink | Log in to Reply

        I chatted with @rmccue, who first introduced WP_HTTP_ResonseInterface (or as previously named WP_JSON_ResponseInterface) on Feb 17, 2014, and he **believes** it can be removed. He has to do some testing to confirm this theory.
        If we can remove it, we will. Otherwise @rmccue will provide the reasoning behind needing to keep it.

    • Weston Ruter 10:46 pm on September 30, 2015 Permalink | Log in to Reply

    • Rachel Baker 2:10 pm on October 2, 2015 Permalink | Log in to Reply

      We have a patch up in https://core.trac.wordpress.org/ticket/33982 that can be refreshed, if committers have any desired code changes.

  • Gary Pendergast 11:17 pm on April 2, 2015 Permalink
    Tags: , , ,   

    OMG EMOJI 😎 

    One of the fun bits of the utf8mb4 upgrade is that we can now store emoji! Once your site is upgraded to utf8mb4, it can natively store any emoji character. There were a couple of problems with that, though:

    • Some browsers don’t know how to render emoji 👎, or they have bugs in their implementation 😢. Notably, Chrome either doesn’t work or has bugs, older versions of IE don’t work, and Firefox has bugs.
    • Not all sites will be able to upgrade to utf8mb4, which means they’ll be unable to save emoji characters that they enter.

    Not being able to use emoji makes everyone a sad panda (😭🐼), so we need to fix this. There are a few moving parts of our emoji support, so lets go through them.

    utf8 backwards compatibility

    If a site can’t be upgraded to utf8mb4, we convert emoji to their HTML-encoded equivalent, and store that, instead. From a UI perspective, post editing works as expected 🎉. Because fields need to be white listed to support this, we don’t allow it everywhere – just post_title, post_content and post_excerpt. We also allow it in the site title and the site description.

    Browser support

    There’s a small compatibility check included on every page, both on the front end, and in the Dashboard. For those interested, this adds 1-4ms (⚡️-fast!) to the page render time – the aim was to keep this as low as possible, to avoid affecting your UX. If you notice a little chunk of compressed JS at the top your HTML, that’s probably it. If you’d like to check out how it works in a more readable format, have a look through wp-emoji-loader.js.

    Email ✉️ and RSS 📯 (There’s no RSS emoji, give me a break.)

    Of course, the JS shim won’t work in email and RSS. So, we replace all emoji with a static PNG version in those cases.

    TinyMCE 📝

    In addition to the browser support JS, there’s a TinyMCE plugin that handles keeping emoji looking good, while you type. It’s pretty magical.

    Taxonomies and URL slugs

    You can totally make taxonomies and URL slugs with emoji in them. Because we love you, and want you to be happy. 😀

    So, that’s about it. If you have any questions about the implementation, drop them in the comments below.

    💩

     
    • Scott Kingsley Clark 11:19 pm on April 2, 2015 Permalink | Log in to Reply

      Does this mean post type names / taxonomy names / option names / meta keys support emoji now?

    • Adam Silverstein 11:24 pm on April 2, 2015 Permalink | Log in to Reply

      👏

    • Lara Littlefield 11:38 pm on April 2, 2015 Permalink | Log in to Reply

      🙌

    • Mustafa Uysal 11:41 pm on April 2, 2015 Permalink | Log in to Reply

      👏

    • Stephen Edgar 11:56 pm on April 2, 2015 Permalink | Log in to Reply

      👯😊👊

    • bravokeyl 12:11 am on April 3, 2015 Permalink | Log in to Reply

      Emoji in URL’s , Cool 1F382

    • Ihor Vorotnov 12:30 am on April 3, 2015 Permalink | Log in to Reply

      omg, will there be a way to prevent upgrading and completely remove this compatibility check and that ‘little chunk of compressed javascript’ which is a thing Google does not tolerate?

      • Gary Pendergast 12:36 am on April 3, 2015 Permalink | Log in to Reply

        There’s a plugin to disable emoji compatibility. It’s currently broken (it’s unhooking the wrong filters), but I assume it’ll be fixed before 4.2 is released.

        which is a thing Google does not tolerate

        Can you please clarify what you mean by that?

        • Ihor Vorotnov 2:53 pm on April 5, 2015 Permalink | Log in to Reply

          Google’s recommendations for faster and better sites include “remove any inline styles and scripts from your html source”. Which is actually a common sense – scripts are scripts, markup is markup. We were fighting for years to remove inline WP gallery styles and now we’ve got inline scripts. Awesome)

          It’s a good thing to include this compatibility in core, but turning emoji support (and adding extra inline script) by default isn’t. Most sites (not blogs) have serious tone of voice and don’t need emoji. Those who want them can turn them on in options or using add_theme_support( ’emoji’ ) for example. That’s how it should be, IMHO, not vice versa.

          • Gary Pendergast 12:16 am on April 6, 2015 Permalink | Log in to Reply

            Google’s recommendations for faster and better sites include “remove any inline styles and scripts from your html source”.

            That is true for large scripts and styles, but not for small scripts.

            For scripts needed for page rendering, Google recommends putting them inline, to avoid extra network requests:

            Scripts that are necessary to render page content can be inlined to avoid extra network requests, however the inlined content needs to be small and must execute quickly to deliver good performance.

            https://developers.google.com/speed/docs/insights/BlockingJS

            • Julie @Niackery 2:32 pm on April 6, 2015 Permalink

              Hi Gary, thanks very much for answering all the questions here! I’m finding the info very useful. I’m just wondering about the second part of the comment above. Currently my settings are not to convert emoji — so as of 4.2 this setting won’t exist anymore? Emoji will be converted automatically regardless of settings? Or will I have to readjust my settings? I really despise the idea of installing a plugin to configure something I should be able to do in my settings. I sincerely hope WordPress isn’t forcing emoji on every single install. As stated above, a great many sites will not appreciate having emoji turned on automatically, especially without the ability to disable without a plugin. WordPress shouldn’t force the use of a plugin to disable such a frivolous feature (which can’t possibly meet the 80% use requirement). Really hoping I’ve misinterpreted this, so I’m very much looking forward to your reply — cheers!

            • Gary Pendergast 1:32 am on April 7, 2015 Permalink

              This is where the difference between smilies and emojis becomes a little more important. The setting you’re referring to is specifically for converting smilies to images. This setting will continue to exist in WordPress 4.2, but will only apply to smilies.

              Emoji are a different case – they’re text, not images. They’re part of the Unicode standard, and all OSes and browsers are working towards supporting them natively. The problem is that the support is not 100% yet, so the emoji image replacement is all about providing a compatibility layer for those users. As native support is rolled out, the compatibility layer will automatically stop loading the image replacement code.

              While I appreciate that your specific usage may not require emoji support, it’s fairly clear from other platforms (including WordPress.com) that emoji are becoming an important part of communication – a simple way to convey a wide range of emotions in a text format. To take an easy example, look at any business page on Facebook – while the business news posts may not use emoji, you’ll struggle to find a comment thread replying to those posts that don’t contain emoji.

              The important part to remember is that, as of WordPress 4.2, all WordPress sites will support emoji. This is entirely a side effect of our improved Unicode support, and it cannot be disabled (in the same way you cannot disable some of the letters in the alphabet). The compatibility layer is focussed on improving the reading experience for your end users. You can disable that if you really want to, but I think the cases for disabling it are fairly narrow.

            • Julie @Niackery 4:05 pm on April 7, 2015 Permalink

              Thanks very much for that answer, Gary! I continued to read up on emoji some more after leaving that comment and suspected I was misunderstanding the issue. Thanks for clearing it all up — I’m assuming/hoping I’m not the only one who may have been confused about this 😉

              I hope you don’t mind my asking one last question — in Facebook or on my phone, I just click on the image of the emoji I want to use (as opposed to typing the symbols for emoticons, as I understand it). So does this mean that the support added to WordPress will also add buttons to the post editor? How would that work for those using the html editor?

              Thanks again for your help!

            • Gary Pendergast 12:37 am on April 8, 2015 Permalink

              Good news! We wrote a codex page, just to let you know how to use emoji:

              https://codex.wordpress.org/Emoji

              (Summary: On your mobile device, your emoji keyboard will work, both in the WordPress app, and if you visit your Dashboard in your Browser. OS X and Windows 8+ also provide emoji keyboards that you can use.)

            • Julie @Niackery 1:14 pm on April 8, 2015 Permalink

              Oh my goodness! You guys are the best 🙂 Thanks a million!

        • Ihor Vorotnov 2:56 pm on April 5, 2015 Permalink | Log in to Reply

          and, btw, check current page’s url in Chrome on Windows.

          • Gary Pendergast 12:17 am on April 6, 2015 Permalink | Log in to Reply

            Windows 7 and earlier has no native rendering of emoji, so native controls (such as the URL bar) won’t render them correctly. Unfortunately, we don’t have any control over that.

    • marsjaninzmarsa 1:38 am on April 3, 2015 Permalink | Log in to Reply

      Ugh, this article in Feedly (via RSS) looks awful. 😞

    • Primoz Cigler 2:06 am on April 3, 2015 Permalink | Log in to Reply

      Hilarious, never seen the emoji in my URL bar before! 😀

    • Brandon Kraft 2:34 am on April 3, 2015 Permalink | Log in to Reply

      Emoji in URL, while supported per spec and fun, isn’t the greatest experience yet. I have a few notes on it toward the end of http://www.brandonkraft.com/b/2015/03/emoji-wordpress-and-you/

      Now that we support it, I would imagine more folks will sooner than later.

    • Nile Flores 3:14 am on April 3, 2015 Permalink | Log in to Reply

      I’d like to use my own emoji like I did back in the day. I’ve got all these emoticon sets I made that I can choose from. Possible to do this like I could in the past??? For example b2 grins by Alex King became wp grins and you could do this for both post types and comments, and then with a a filter, change the path to the directory of the emoticons you want to use if you wanted to use your own.

      What about like the old days when the comment form on posts, you saw the emoji load and just clicked to add them into your comment?

      • Gary Pendergast 3:40 am on April 3, 2015 Permalink | Log in to Reply

        You can absolutely use a different emoji set!

        As this is based on Twemoji, it uses their naming scheme for the image files. For single character emoji, that looks like “1f4a9.png” (💩). For two character emoji, that looks like “1f1ec-1f1e7.png” (🇬🇧). EmojiOne, for example, uses the same naming scheme, so a plugin to convert emoji to EmojiOne would simple use the “emoji_url” filter to change the location of the emoji images.

        We won’t be doing an emoji selector in core, though. It would be about 400KB of JSON, plus the images – pretty weighty to include on every page. 🙂

        • Nile Flores 11:06 am on April 3, 2015 Permalink | Log in to Reply

          Yay! That makes me a happy panda… lol

        • marsjaninzmarsa 9:33 pm on May 1, 2015 Permalink | Log in to Reply

          As this is based on Twemoji, it uses their naming scheme for the image files. For single character emoji, that looks like “1f4a9.png” (💩). For two character emoji, that looks like “1f1ec-1f1e7.png” (🇬🇧). EmojiOne, for example, uses the same naming scheme, so a plugin to convert emoji to EmojiOne would simple use the “emoji_url” filter to change the location of the emoji images.

          You’ve inspired me. 😀
          https://github.com/marsjaninzmarsa/WordPress-Emoji-Might

    • Dave McHale 1:00 pm on April 3, 2015 Permalink | Log in to Reply

      As a chrome user, this does little to excite me. Rectangles in the URL, yay? My email notification about this post was LITTERED with rectangles in Gmail, though most emoji seem to render fine on the page itself. It seems like there are too many rendering support issues for this still… My $0.02

      • Pascal Birchler 5:37 pm on April 3, 2015 Permalink | Log in to Reply

        I use Chrome as well and the emoji in the URL is displayed just fine. And I’m sure Gmail supports emojis just fine too and the rectangles in the email were caused by the Jetpack email subscription.

      • Gary Pendergast 10:37 pm on April 3, 2015 Permalink | Log in to Reply

        Unfortunately, we can’t do anything about Chrome’s URL bar – that only works where the OS has native support for emoji.

        The email problem will be fixed next week: Jetpack’s email subscriptions go through WordPress.com, which I haven’t rolled full emoji support out to, yet. It’s on my list. 😎

    • Chris Van Patten 10:18 am on April 5, 2015 Permalink | Log in to Reply

      I noticed that in Chrome 42.0.2311.68 beta on OS X, the script still kicks in and replaces the emoji even though this version of Chrome fully supports emoji. There’s a noticeable flash as the system-standard emoji get replaced with the Twemoji variants.

      Is that intentional? I thought the aim was for the script to detect if the browser already supports emoji and only kick in if it doesn’t.

      • Gary Pendergast 10:24 am on April 5, 2015 Permalink | Log in to Reply

        This is intentional. Chrome OS X doesn’t render colour emoji if the font weight of the element they’re in is greater than 500 (ie, anything bolder than normal). So, rather than have invisible emoji under some circumstances, we’re replacing all emoji on the page. Once Chrome releases a version that does render emoji in bold elements, the script will automatically stop loading Twemoji.

        For reference, here’s the Chrome ticket for this bug: https://code.google.com/p/chromium/issues/detail?id=441946

    • Mike Crantea 5:23 pm on April 23, 2015 Permalink | Log in to Reply

      I noticed that when changing the option in the admin from Writing to not convert emoticons to graphics, the inline styles and the emoji JS still comes through. I believe they should be ignored when the site owner does not want to use this feature.

      • Ipstenu (Mika Epstein) 11:58 am on April 24, 2015 Permalink | Log in to Reply

        Comes through on the editor or on the pages people visit?

        Unlike smiles, one can enter emoji into text without using emoticons like 🙂

        This, for example, is emoji — 😐

        This is a smilie: 😀

        So one can have both, one, or none

        • Monika 10:44 am on April 25, 2015 Permalink | Log in to Reply

          “Comes through on the editor or on the pages people visit?”
          The scripts are still at wp_head, if I disable “convert smilies” or not, this is frustrating.
          And my old smilies are broken. 😥 and so on… 🙁

  • Gary Pendergast 2:25 am on April 2, 2015 Permalink
    Tags: , , utf8mb4,   

    The utf8mb4 Upgrade 

    In WordPress 4.2, we’re upgrading tables to utf8mb4, when we can. Your site will only upgrade when the following conditions are met:

    • You’re currently using the utf8 character set.
    • Your MySQL server is version 5.5.3 or higher (including all 10.x versions of MariaDB).
    • Your MySQL client libraries are version 5.5.3 or higher. If you’re using mysqlnd, 5.0.9 or higher.

    The difference between utf8 and utf8mb4 is that the former can only store 3 byte characters, while the latter can store 4 byte characters. In Unicode terms, utf8 can only store characters in the Basic Multilingual Plane, while utf8mb4 can store any Unicode character. This greatly expands the language usability of WordPress, especially in countries that use Han character sets. Unicode isn’t without its problems, but it’s the best option available.

    utf8mb4 is 100% backwards compatible with utf8.

    Due to index size restrictions in MySQL, this does mean we need to re-create a handful of indexes to fit within MySQL’s rules. Using a standard configuration, MySQL allows 767 bytes per index, which for utf8 means 767 bytes / 3 bytes = 255 characters. For utf8mb4, that means 767 bytes / 4 bytes = 191 characters. The indexes that will be resized are:


    wp_usermeta.meta_key
    wp_terms.slug
    wp_terms.name
    wp_commentmeta.meta_key
    wp.postmeta.meta_key
    wp_posts.post_name

    And from Multisite:


    wp_site.domain
    wp_sitemeta.meta_key
    wp_signups.domain

    Of course, the Multisite (and wp_usermeta) keys obey the DO_NOT_UPGRADE_GLOBAL_TABLES setting. The upgrade will only be attempted once, though we’ll probably add a check in a future WordPress version to see if we can upgrade now (say, if you’ve upgraded your MySQL server since upgrading to WordPress 4.2).

    If you’re a plugin developer and your plugin includes custom tables, please test that your indexes fit within MySQL’s limits. MySQL won’t always produce an error when the index is too big, so you’ll need to manually check the size of each index, instead of relying on automated testing.

    EDIT: One more thing…

    If you’d like to upgrade your custom tables to utf8mb4 (and your indexes are all in order), you can do it really easily with the shiny new maybe_convert_table_to_utf8mb4( $tablename ) function. It’s available in `wp-admin/includes/upgrade.php`, and will sanity check that your tables are entirely utf8 before upgrading.

     
    • Thomas Townsend 2:33 am on April 2, 2015 Permalink | Log in to Reply

      Hi Gary, glad to see you and the team are looking after the devs too ? I owe you a beer or tow if you even get down to Tampa Florid area…

    • Pippin Williamson 2:36 am on April 2, 2015 Permalink | Log in to Reply

      Excellent, thanks for the notice!

    • Todd Lahman 3:47 am on April 2, 2015 Permalink | Log in to Reply

      Great to see WP Keeps moving forward. Thanks for the update Gary.

    • J.D. Grimes 1:03 pm on April 2, 2015 Permalink | Log in to Reply

      @pento I’ve noticed that the difference in index length can cause notices when updating tables using `dpDelta()`.

      I was getting this error locally, when running the install tests for a plugin against WordPress trunk:

      `
      WordPress database error: [Duplicate key name ‘meta_key’]
      ALTER TABLE wptests_dbdelta_test3 ADD KEY meta_key (meta_key)
      `

      After some debugging, I found that dbDelta() was being thrown off by KEYs for VARCHAR columns, because the description of the key from the database would be like this:

      `KEY meta_key (meta_key(191))`

      But in my CREATE TABLE string, they are defined like this:

      `KEY meta_key (meta_key)`

      Since these don’t match `dbDelta()` attempts to add the key again, resulting in the error. This would happen to users on sites that support utf8mb4, if the plugin is deactivated and then reactivated.

      Kind of related: #31388.

      • Gary Pendergast 1:29 pm on April 2, 2015 Permalink | Log in to Reply

        Thanks for the bug report, @jdgrimes! I’ve recorded it in #31869, I’ll have a think about the best way to fix it.

        In the mean time, a valid work around is to specify the index length in your CREATE TABLE statement – this is what we’ve done in core, hence why I haven’t run into this bug previously.

    • pix365 3:05 pm on April 13, 2015 Permalink | Log in to Reply

      For folks on v4.1.1 ior earlier – Will there be a plugin released that will perform the the sanity checks to ensure existing sites have all the SQL pre-requisites in place before folks even attempt the upgrade to 4.2. ( or have miss understood the above “Edit on the upgrade.php” or is this check is already built in to 4.1.1)

      • Ipstenu (Mika Epstein) 4:00 pm on April 13, 2015 Permalink | Log in to Reply

        The checks are built in.

        > Your site will only upgrade when the following conditions are met:…

        That’s what’s doing it 🙂 If any condition fails, no DB changes. If you’re not even on the right version of MySQL, it won’t do it either.

    • snowboardmommy 12:30 am on April 24, 2015 Permalink | Log in to Reply

      FWIW, I upgraded a multisite installation (with debug mode ON) to 4.2 today and also found a similar DB error message in the error_log in wp-admin:

      WordPress database error Duplicate key name ‘meta_key’ for query ALTER TABLE wp_usermeta ADD KEY meta_key (meta_key(191)) made by wp_upgrade, make_db_current_silent, dbDelta
      WordPress database error Duplicate key name ‘domain’ for query ALTER TABLE wp_site ADD KEY domain (domain(140),path(51)) made by wp_upgrade, make_db_current_silent, dbDelta
      WordPress database error Duplicate key name ‘meta_key’ for query ALTER TABLE wp_sitemeta ADD KEY meta_key (meta_key(191)) made by wp_upgrade, make_db_current_silent, dbDelta
      WordPress database error Duplicate key name ‘domain_path’ for query ALTER TABLE wp_signups ADD KEY domain_path (domain(140),path(51)) made by wp_upgrade, make_db_current_silent, dbDelta

      I haven’t peeked at the database (phpMyAdmin) yet, but everything appears to be working alright.

    • jalev 8:01 am on April 24, 2015 Permalink | Log in to Reply

      Hi Gary, I did the upgrade to 4.2 and afterwards it asks me for a database upgrade as I obviously belong to one of the cases you described above. The problem is that the Database Update process timeouts.

      I have increased the timeout to 60 minutes but there is an alter query that after almost 15 minutes of running, it crashes my server as it takes all the ram and swap available. The query is the following: ALTER TABLE nB8KH50Wbb_options CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci

      and below it there is a huge que of locked queries as they try to write in the same table “Waiting for table metadata lock”.

      Any ideas?

      • Ipstenu (Mika Epstein) 12:06 pm on April 24, 2015 Permalink | Log in to Reply

        How big is your db? How big are your post and post meta tables?

        • FolioVision 8:31 am on May 7, 2015 Permalink | Log in to Reply

          Hello Mika,

          we had the same issue, we had to do this with a command line PHP, see our link in other comment here.

          Here is some info about the two sites where we had the issue:

          Site 1: wp_posts 256MB, wp_postmeta 57MB, wp_comments 292MB, wp_commentmeta 181.9MB, wp_options 19.8MB
          Site 2: wp_posts 84.9MB, wp_postmeta 5MB, wp_comments 134MB, wp_commentmeta 173.7MB, wp_options 9MB

          Perhaps you should improve the upgrade process to remember which queries finished before it times out. When it times out, it seems to be executing all the queries again, even if they finished before.

          Thanks,
          Martin

          • jstensved 4:26 pm on May 22, 2015 Permalink | Log in to Reply

            I can confirm that I’m having the exact same issue on another large site which I can’t update. I’m updating from command line with WP-CLI but still the command will eventually consume all memory.

    • l3hworks 4:00 pm on April 24, 2015 Permalink | Log in to Reply

      Hello, I updated to 4.2 in xampp and now I can’t go back to my life server (mysql 5.1), is there any way I could revert this?

    • Nihad 4:46 pm on April 24, 2015 Permalink | Log in to Reply

      I’ve upgraded to 4.2 today, on two different domains, one is MultiSite (Danish) other one (in English) is not.
      I’ve got no errors, so all is good. But my encoding is wrong. Content is not being displayed properly, even if it is created and saved as UTF8. On both sites.

      Since then, in troubleshooting, I’ve manually converted my db/data/tables from UTF8 to UTF8MB4, to see if this will resolve the issue. Nope.
      Iconv utility converting data to UTF8 just in case it was not. Did nothing either.
      My data looking at it directly in mysql looks good. Even Sequel Pro displays data correctly.

      It worked perfectly on pre 4.2… now it does not.
      Posts that are incorrectly displayed are non editable in WordPress either. It show, empty titles and contents. and when you save them, they are empty. So it’s like it can’t recognize encoding and then it just saves it as empty.

      But… If i copy data directly from database, paste is into a post, that is displayed as empty, in WordPress editor, and save it… It works correctly. So already existing data is somehow read incorrectly, but if the same data is copied into the editor and save to database it is good.

      Seems like there is an issue, in updating of database that went wrong somewhere.

      • sistemasBebetter 6:51 pm on April 29, 2015 Permalink | Log in to Reply

        Have you found any solution to this? I’m having the same problem but only with one of the sites I have (the others in the same computer and database upgrade correctly).
        Which WP version had your sites before upgrading?

      • watomsk 7:50 am on May 3, 2015 Permalink | Log in to Reply

        Did you manage to find a solution to this? I upgraded to 4.2 and have the same problem with my website.

    • neo7 12:58 pm on April 26, 2015 Permalink | Log in to Reply

      Got this error while updating “PHP message: WordPress database error Specified key was too long; max key length is 1000 bytes for query ALTER TABLE wp_commentmeta CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci made by wp_upgrade, upgrade_all, upgrade_420, maybe_convert_table_to_utf8mb4”

      Now all tables are utf8mb4_unicode_ci except wp_commentmeta which is utf8_general_ci.

      How to covert this table?

    • interglobalmedia 1:47 am on April 27, 2015 Permalink | Log in to Reply

      Hi Nihad. I had kind of a similar issue when I updated to 4.2. What is actually happening is that incorrect code that was previously undetected or slipped under the radar so to speak, has now been detected and then “erased”. I’ll give an example. I was working with a theme that someone else had begun developing, and there was incorrect code in it. I hadn’t gone over everything yet, and did not realize that the theme’s sidebar had not been registered or configured properly. So, when I updated to 4.2, I was told in the backend that my sidebar’s id had been incorrectly configured. Could I please manually correct it? I forget exactly what I did, but when I did it, the sidebar disappeared. What was good about this and I saw as a real plus, was that it told me what was wrong, and then I knew what I had to do to fix it. i fixed it. Luckily, it was on a local install. Of course. I was still developing it. Then I had another issue post 4.2 that had never come to the fore previously. I have a theme that uses a page template called “full width content”. However, I had purchased a premium plugin called “Ultimate Landing Pages” back in February. It wasn’t quite clear what sort of template the developer used for these landing pages, but everything seemed to work just great. Until 4.2. Today, I went into one of my site’s regular pages that I had configured to “full width content a couple of months ago, and I noticed that it no longer ran the full page width that it had before 4.2! The now invisible sidebar prevented the content from running the full width of the page. I thought it bizarre and went into the Editor to see what was going on. The Ultimate Landing Pages plugin’s template finally had shown up. It was configured to the “Front Page Template, which is one of the WordPress “Reserved Theme File Names”, has special meaning for WordPress, and if present along with a custom theme template that does the same thing, it will take precedence. So the fact that I had both the Front Page template and the Full Width template which both displayed the page in the same way, caused the Full Width template to no longer display “properly”. I had to go into the Editor and change the page template to “Front Page”. This partially has to do with cache issues, but also 4.2 lets you know either overtly (or covertly) about incorrect code/configuration issues. I am going to try and find out from the WordPress core developers what changes they made that made this possible. I see it as a very positive thing. Hope this helps.

    • DrLightman 10:40 am on April 27, 2015 Permalink | Log in to Reply

      Could this index upgrade fail or time out on databases with large tables, let say 250’000+ posts thus lot more postmetas?

      If we do this upgrade manually let say via command line on dedicated servers, will WP 4.2 skip the upgrade during installation?

    • jtleathers 9:56 pm on April 27, 2015 Permalink | Log in to Reply

      Developing a site locally on WAMP with SQL 5.6.17. When I attempt to import the database to a server using an older version of SQL I get the following error:

      #1273 – Unknown collation: ‘utf8mb4_unicode_ci’

      How do I properly rollback this “upgrade” from WordPress 4.2?

      • Ipstenu (Mika Epstein) 11:30 pm on April 27, 2015 Permalink | Log in to Reply

        Import how? Phpmyadmin?

        • mctenold 11:55 pm on April 27, 2015 Permalink | Log in to Reply

          Same issue here – importing any way, CLI, phpMyAdmin, or personally I’m using MySQL Workbench for Mac and getting the same errors. For now I’m going to have to not use 4.2 until I can find a workflow to go from Local -> Dev Server, where local MySQL is compatible with 4.2 but dev server is not.

          • Max Sixblade 5:16 am on May 4, 2015 Permalink | Log in to Reply

            True! Using XAMPP for Windows, this is definitely not a phpmyadmin issue. Older mysql version on remote server leads to errors while importing db. Compatibility modes while exporting db didn’t help in my case.

        • jtleathers 12:45 am on April 28, 2015 Permalink | Log in to Reply

          Yeah, import through PHPMyAdmin.

        • MBWD 3:23 pm on May 16, 2015 Permalink | Log in to Reply

          I am having the same problem with my shared hosting server, which is using an older version of PHP. The upgrade to 4.2 apparently went fine for sites already hosted, but importing a db with utf8mb4 doesn’t work. I tried downgrading my version of WP to 4.0 and upgrading the database (which I thought would revert the charset) but it didn’t work. Manually replacing didn’t work either.

    • mctenold 12:54 am on April 28, 2015 Permalink | Log in to Reply

      I’m in the same boat as @jtleathers, my typical workflow is to install WordPress locally for a new site, build the site locally, and then move everything to a dev or production server once I’m at a good place.

      The issue comes in when my client’s dev or production server has a MySQL version 5.5.3. I get the same error upon attempting to import the database: Unknown collation: ‘utf8mb4_unicode_ci’.

      This seems like a misstep of an upgrade, I think my workflow is pretty common practice and will affect a lot of people. I don’t think it’s common to install/develop the site on the production server.

      Any remedy for my dilemma?

      Seems like the only possible fix right now is to downgrade my local MySQL version, or not use WordPress 4.2.

      🙁

    • dustinbolton 10:20 pm on May 6, 2015 Permalink | Log in to Reply

      BackupBuddy developer here: It is absolutely INCORRECT to say that migrating sites to new server is uncommon. BackupBuddy helps users move several hundred thousand sites per year to new servers.

      We are already seeing a flood of users encountering this problem attempting to move their site to new servers with older MySQL versions. They of course are puzzled why this is a problem since WordPress states it supports this older MySQL version.

      Many users these days practice things such as local development, staging, etc which are good modern practices. WordPress has effectively put a huge thorn in the sides of developers and in my opinion has made an extremely big mistake.

    • lucaboccianti 6:38 pm on May 11, 2015 Permalink | Log in to Reply

      I may have found a possible walkaround.

      I’m in the same situation and problem as above: develop locally with xampp then upload on the production server (mostly shared hosting).

      with phpmyadmin for each table edit each field codified with utf8mb4-something to be codified as utf8-something (select the table, then Structure, etc.). then edit the table properties (Operations) and do the same.

      finally export the dump and restore it on the production server.

      it’s long and painful but maybe better than restart a project from scratch.

    • lucaboccianti 6:49 pm on May 11, 2015 Permalink | Log in to Reply

      problems: you’ll end with all the extended characters looking funny and apparently the text edit area appear blank even if the text is visible on the frontend.

    • Max Sixblade 11:36 pm on May 11, 2015 Permalink | Log in to Reply

      So.. where shall we vote to get that “install in default UTF-8” checkbox on WP installation?)

    • Ben Lobaugh (blobaugh) 1:44 pm on May 12, 2015 Permalink | Log in to Reply

      I have had several migrations happen this week, and even between the dev, staging, and production servers. Ran into lots of issues and what I wound up doing is running a script to convert the database back to utf8. This worked great to get the data imported and the next time wp-admin was loaded the database was successfully updated with the security patch.

      If interested here is the script I used http://ben.lobaugh.net/blog/201740/script-to-convert-mysql-collation-from-utf8mb4-to-utf8

      • dblomster 3:23 pm on May 12, 2015 Permalink | Log in to Reply

        Nice! What about table indexes and such? Is there anything else that need to be changed when going from utf8mb4 to utf8 again?

        • Ben Lobaugh (blobaugh) 4:54 pm on May 12, 2015 Permalink | Log in to Reply

          All I did was run that and it magic worked. Before running that script the db was complaining when I tried to import via sql files.

        • MBWD 4:00 pm on May 16, 2015 Permalink | Log in to Reply

          Hi Ben, I’m also on a Mac and have saved this script you so thoughtfully provided as a .sh file to run through Terminal, but when I run it I am getting an error saying “mysql: command not found”. Any idea what could be wrong?

      • dotwongdotcom 4:40 pm on May 12, 2015 Permalink | Log in to Reply

        Ben! Thanks for this! You may have just saved me a huge headache. I stumbled upon your post through a rabbit hole of clicking while trying to figure out a workaround for errors that I was encountering while using the migration tool BackupBuddy. My local dev is running WP 4.2, but the migration won’t allow utf8mb4 on the live site because their version of mysql. I’m a relative novice when it comes to these things… how would I implement your script to convert my local build’s tables back to utf8 so that I can migrate to my live server?

        • Ben Lobaugh (blobaugh) 9:09 pm on May 12, 2015 Permalink | Log in to Reply

          Copy it into a bash script on your local dev. Edit the variables for db access and run it from the terminal. This assumes you are on a unix based system such as Linux or OS X. I do not have Windows to know how to build a script for that.

    • zanozik 1:38 am on June 23, 2015 Permalink | Log in to Reply

      Server version: 5.5.19-cll
      MySQL client version: 5.5.33
      Upgraded to 4.2.2 and my wp “crashed”. Every option that was in unicode was either unavailable or missing non-latin chars. Pages and posts content was unviewable. Titles were gibberish.
      Since I use a web hosting (no root, or cpanel access) with remote phpmyadmin I logged in there to see that all of the tables collation have been changed to utf8mb4-general. Database collation was still utf8-general, strange, I thought.
      The hosting I use, let a user create/change a database in custom interface only. Rather popular solution. So I thought maybe the upgrade was only partly successful and the database was not charset was not actually upgraded, just collation changed. So I changed charset in wp-config, changed collations for each tables, and voalia – everything working again…
      Could that buggy force upgrade happen to “omg-so-professional” WordPress? I just dont know what to do, laugh or cry right now…

    • physalis 11:20 pm on July 2, 2015 Permalink | Log in to Reply

      @zanozik I had the exact same problem, only when I changed from a dev to the final server – every little special character was nuked, plus options and posts in the backend where cracked. I tried setting my dev server database, but to no avail (so I already thought it must be the files then).
      Your description helped me save a release date for that project tomorrow morning and thus quite some people around it ;). So thanks a bunch!

    • treavioli 8:34 am on July 10, 2015 Permalink | Log in to Reply

      I encountered this issue when I tried to export my database tables via phpmyadmin. I would get several errors and warnings about collation. I switched to Sequel Pro and exported/imported that way, but I’m still not without issues. My post editor won’t recognize posts/pages/etc I created before site migration. Nothing appears in the edit panel for either Visual or Text tab..

  • Gary Pendergast 1:26 pm on February 25, 2015 Permalink
    Tags: , ,   

    Emoji Chat Agenda, February 25, 2015 

    Here’s the agenda for Wednesday’s Emoji Chat in the #core channel on Slack.

    Time/Date: Immediately after the Dev Chat.

    1. Emoji Helper – On platforms that don’t have an emoji keyboard, should we provide one?
    2. Performance – When we’re falling back to Twemoji on large pages, we have to consider how it will perform. Are there faster ways of replacing the emoji characters with images?
    3. Open Floor – There’s still time to rant about the evils of emoji! Let us know how you really feel.
     
  • Gary Pendergast 12:26 am on February 13, 2015 Permalink
    Tags: ,   

    Emoji Chat Meeting Notes, February 12, 2015 

    The full meeting archive is available here.

    1. Why we’re doing this

    So, here’s a bit of back story.

    As of r31349, WordPress partially supports emoji. ~60% of WordPress sites are running MySQL 5.5 or later (so can be upgraded to store emoji), and ~40% of browsers natively support emoji. Emoji are a wildly popular method of communication, so we can expect them to be heavily used as soon an they’re available. The problem is, 60%/40% means a really bad experience for a huge number of our users, who’ll try to use emoji, and fail.

    This is where the emoji feature plugin comes in to play. In order to help the 40% of WordPress sites that can’t be upgraded to store emoji natively, the wp_encode_emoji() function will turn them into HTML entities. Due to the unimaginable joy that character sets brings me, this will only be applied to sites using the utf8 character set, which accounts for the vast majority of WordPress sites – utf8 has been the default character set since r4860.

    To help the 60% of browsers that don’t display emoji natively, we’re using the Twemoji image set as a fallback. This lets us show emoji everywhere, without causing extra load where emoji are already supported, mobile browsers being the important example here.

    Now, there have been some concerns brought up previously that I’d like to address.

    “Is this really appropriate for core?”

    Yes. (Obviously that’s my answer, or we wouldn’t be here.) WordPress is is the business of making communication simple and accessible for all. Tech users everywhere have clearly chosen emoji as a means of communication, so it’s up to us to make sure they can do that within WordPress as easily as possible, or risk being left behind.

    “Should we be concerned about changing the images in the future? Wouldn’t we be altering users’ content?”

    No. By using Twemoji only when we can’t provide native support for emoji, it’s a pretty clear message that while the general appearance of emoji stays the same, the actual sprite used can differ significantly between platforms. (For example, every emoji set except Android uses a left hand for :thumbsup:.) As more browsers add native support for emoji, Twemoji usage will drop, reducing even further any impact we can have on users.

    And so, that brings us to today.

    2. The current state of the plugin

    The plugin is very close to done. The editor plugin needs some attention, which @azaozz will be providing soon. There are a few bugs to discuss, which are mostly around fallback behaviour in browsers that don’t support emoji natively. Apart from that, the basic functionality is pretty much how I would expect it to appear in core. It’s had a brief review from the accessibility team, with only some minor alterations needed. The Twemoji images won’t be included in wordpress.zip, as it’s a total of 3.4MB of images. They’re currently hosted on WP.com’s CDN, but we’re investigating other options for where to host them, probably the W.org CDN. Given that the wp-admin Dashboard also loads things from Google, I have no problem with hosting them on an external CDN. There will naturally be a filter on the URL, to allow local hosting for sites that don’t want to use the CDN.

    One of the major concerns at the moment is that we’re going to be splitting data formats, depending on if the site uses the utf8mb4 or the utf8 character set. utf8mb4 stores emoji natively, while utf8 requires us to HTML encode the emoji characters. In the futures, we’ll look at upgrading sites to utf8mb4 if they’ve upgraded their MySQL since WordPress 4.2, but that leaves the potential for mixed encoding – old posts having HTML encoding, new posts having the native characters. A post will be automatically updated to native upon saving, but do we need to consider upgrade routines, to go through all old posts and convert them?

    Export/import also needs thorough testing, particularly when importing and exporting between sites having different character sets.

    3. Unicode 8.0: the future of emoji

    To talk about the future of emoji, you need to know a little bit of history. At the basic level, emoji are all single characters defined within the Unicode standard. However, they also support modifiers. Modifiers are a second character following the first, which usually causes the two characters to be merged into a single character when rendered. A good example of this is flag emoji.

    The character G is U+1F1EC. The character B is U+1F1E7 (these characters are different to their ASCII equivalent). When used individually, they’ll display as that letter. When combined next to each other, they’ll display as the British flag.

    So, Unicode 8.0 will two interesting things: a set of 37 new emoji, and skin tone modifiers. When a skin tone modifier character is placed after any face or person emoji, the emoji will show with that skin tone. Unicode 8.0 is due to be finalised in August 2015, so we and (Twemoji) will be looking at adding support for these then.

    From a technical perspective, it just means we need to be aware that emoji are not always one character, and the methods for detecting multi-character emoji are about to get more complex.

    We’ll also be able to detect if a browser is able to render the new emoji and skin tones, and fall back to Twemoji if they can’t. I don’t have a timeline for when browsers will support the new emoji, so I think it’d be good for us to get ahead of the curve then.

    utf8mb4 stores anything in the Unicode address space, including unallocated characters, so I don’t expect any problems with storage of new emoji.

     
    • Ryan Hellyer 8:45 am on February 18, 2015 Permalink | Log in to Reply

      I’m not seeing any reason to host them externally. Why not just bundle an extra 3.4 MB into the zip?

      Even better, perhaps this should just stay as a plugin? Maybe I’m wrong, but this seems a very niche thing to me.

      • John James Jacoby 1:17 pm on February 18, 2015 Permalink | Log in to Reply

        Particularly in the US, I can anecdotally tell you that Emoji has had slow adoption for several years, but it’s becoming less-so with the native inclusion of it in iOS. See my comment below about smilies, which also very easily could have been a plugin (and may have originally been at one point?) but they’re so simple and commonplace now, it’s weird not seeing them rendered as images. 🙂

      • Gary Pendergast 8:26 pm on February 18, 2015 Permalink | Log in to Reply

        Why not just bundle an extra 3.4 MB into the zip?

        They’re compressed PNGs, so they won’t compress much further. They’d be doubling the size of the zip.

        Maybe I’m wrong, but this seems a very niche thing to me.

        Not so much. Emoji are a very popular method of communication by large groups of the the population. If anything, I’d say it’s an oversight that it’s taken us so long to get to supporting emoji.

    • John James Jacoby 1:15 pm on February 18, 2015 Permalink | Log in to Reply

      Something I don’t see mentioned is the current iteration of smilies.

      • Does Emoji support secretly allow us to nix old-school smilies and cut out an option?
      • Can/should Emoji support be tacked on-top-of smilies and work similarly to Slack and use a bunch of string-replacements?

      Understanding the approach and code behind each is different, they are identical to a typical end-user because – rather conveniently for us – smilies in core have no UI (though there are plugins that implement one.)

      I’m imagining either a new UI option for Emoji similar to smilies in writing settings, or having it use that same one, which led me to wondering how dissimilar the underlying functionality needs to be, or if this is an opportunity to update an old API.

      • Gary Pendergast 8:33 pm on February 18, 2015 Permalink | Log in to Reply

        Does Emoji support secretly allow us to nix old-school smilies and cut out an option?

        If I told you, it wouldn’t be a secret anymore, would it? 😉

        Can/should Emoji support be tacked on-top-of smilies and work similarly to Slack and use a bunch of string-replacements?

        Nope. That’s a pretty niche usage, with native OS and browser support for emoji increasing, I’d prefer to keep that kind of hack out of core. It’d also be a super expensive search/replace to have happen at render time.

        Currently, there’s no plan to change/remove the existing smiley option – emoji will just work if you happen to use them, smilies will continue to be optional (but will be replaced by nicer images if you do use them).

        I would like to deprecate the smiley option at some point, but I haven’t given it enough thought to say when that could/should happen.

    • Paal Joachim Romdahl 11:54 pm on February 18, 2015 Permalink | Log in to Reply

      It seems adding emoji to core will cause some perhaps a lot of frustrations for people:
      http://wptavern.com/wordpress-4-2-on-track-to-expand-core-support-for-emoji

      As the comment seems to be they can add that feature to core but what about…..-insert feature-

      • Paal Joachim Romdahl 1:24 am on February 20, 2015 Permalink | Log in to Reply

        Another thought….. Call it a Easter Egg. Like in games that have hidden features. Press a combination of keys or go and do something special in a game and it unlocks something cool.
        I think each release of WordPress could include a Easter Egg (“just” for fun and unique use). Emoji is the first Easter Egg and fits perfectly as Easter is around the time of the 4.2 release.

        So when people ask why it was included in WordPress 4.2 one can say it is an easter egg. Meaning it was included “just” for fun (of course it has it purpose for being included).

  • Gary Pendergast 11:25 pm on February 11, 2015 Permalink
    Tags: ,   

    Emoji Chat Agenda, February 12, 2015 

    Here’s the agenda for Thursday’s Emoji Chat in the #core channel on Slack.

    Time/Date: February 12 2015 23:00 UTC

    1. Why we’re doing this – @pento
    2. The current state of the plugin – @pento
    3. Unicode 8.0, and future plans – @pento
    4. Open Floor/ranting about the evils of emoji – everyone

    See you tomorrow!

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel
Skip to toolbar