Make WordPress Core

Updates from Gary Pendergast Toggle Comment Threads | Keyboard Shortcuts

  • Gary Pendergast 11:17 pm on April 2, 2015 Permalink |
    Tags: , , ,   

    OMG EMOJI 😎 

    One of the fun bits of the utf8mb4 upgrade is that we can now store emoji! Once your site is upgraded to utf8mb4, it can natively store any emoji character. There were a couple of problems with that, though:

    • Some browsers don’t know how to render emoji 👎, or they have bugs in their implementation 😢. Notably, Chrome either doesn’t work or has bugs, older versions of IE don’t work, and Firefox has bugs.
    • Not all sites will be able to upgrade to utf8mb4, which means they’ll be unable to save emoji characters that they enter.

    Not being able to use emoji makes everyone a sad panda (😭🐼), so we need to fix this. There are a few moving parts of our emoji support, so lets go through them.

    utf8 backwards compatibility

    If a site can’t be upgraded to utf8mb4, we convert emoji to their HTML-encoded equivalent, and store that, instead. From a UI perspective, post editing works as expected 🎉. Because fields need to be white listed to support this, we don’t allow it everywhere – just post_title, post_content and post_excerpt. We also allow it in the site title and the site description.

    Browser support

    There’s a small compatibility check included on every page, both on the front end, and in the Dashboard. For those interested, this adds 1-4ms (⚡️-fast!) to the page render time – the aim was to keep this as low as possible, to avoid affecting your UX. If you notice a little chunk of compressed JS at the top your HTML, that’s probably it. If you’d like to check out how it works in a more readable format, have a look through wp-emoji-loader.js.

    Email ✉️ and RSS 📯 (There’s no RSS emoji, give me a break.)

    Of course, the JS shim won’t work in email and RSS. So, we replace all emoji with a static PNG version in those cases.

    TinyMCE 📝

    In addition to the browser support JS, there’s a TinyMCE plugin that handles keeping emoji looking good, while you type. It’s pretty magical.

    Taxonomies and URL slugs

    You can totally make taxonomies and URL slugs with emoji in them. Because we love you, and want you to be happy. 😀

    So, that’s about it. If you have any questions about the implementation, drop them in the comments below.


    • Scott Kingsley Clark 11:19 pm on April 2, 2015 Permalink | Log in to Reply

      Does this mean post type names / taxonomy names / option names / meta keys support emoji now?

    • Adam Silverstein 11:24 pm on April 2, 2015 Permalink | Log in to Reply


    • Lara Littlefield 11:38 pm on April 2, 2015 Permalink | Log in to Reply


    • Mustafa Uysal 11:41 pm on April 2, 2015 Permalink | Log in to Reply


    • Stephen Edgar 11:56 pm on April 2, 2015 Permalink | Log in to Reply


    • bravokeyl 12:11 am on April 3, 2015 Permalink | Log in to Reply

      Emoji in URL’s , Cool 1F382

    • Ihor Vorotnov 12:30 am on April 3, 2015 Permalink | Log in to Reply

      omg, will there be a way to prevent upgrading and completely remove this compatibility check and that ‘little chunk of compressed javascript’ which is a thing Google does not tolerate?

      • Gary Pendergast 12:36 am on April 3, 2015 Permalink | Log in to Reply

        There’s a plugin to disable emoji compatibility. It’s currently broken (it’s unhooking the wrong filters), but I assume it’ll be fixed before 4.2 is released.

        which is a thing Google does not tolerate

        Can you please clarify what you mean by that?

        • Ihor Vorotnov 2:53 pm on April 5, 2015 Permalink | Log in to Reply

          Google’s recommendations for faster and better sites include “remove any inline styles and scripts from your html source”. Which is actually a common sense – scripts are scripts, markup is markup. We were fighting for years to remove inline WP gallery styles and now we’ve got inline scripts. Awesome)

          It’s a good thing to include this compatibility in core, but turning emoji support (and adding extra inline script) by default isn’t. Most sites (not blogs) have serious tone of voice and don’t need emoji. Those who want them can turn them on in options or using add_theme_support( ’emoji’ ) for example. That’s how it should be, IMHO, not vice versa.

          • Gary Pendergast 12:16 am on April 6, 2015 Permalink | Log in to Reply

            Google’s recommendations for faster and better sites include “remove any inline styles and scripts from your html source”.

            That is true for large scripts and styles, but not for small scripts.

            For scripts needed for page rendering, Google recommends putting them inline, to avoid extra network requests:

            Scripts that are necessary to render page content can be inlined to avoid extra network requests, however the inlined content needs to be small and must execute quickly to deliver good performance.


            • Julie @Niackery 2:32 pm on April 6, 2015 Permalink

              Hi Gary, thanks very much for answering all the questions here! I’m finding the info very useful. I’m just wondering about the second part of the comment above. Currently my settings are not to convert emoji — so as of 4.2 this setting won’t exist anymore? Emoji will be converted automatically regardless of settings? Or will I have to readjust my settings? I really despise the idea of installing a plugin to configure something I should be able to do in my settings. I sincerely hope WordPress isn’t forcing emoji on every single install. As stated above, a great many sites will not appreciate having emoji turned on automatically, especially without the ability to disable without a plugin. WordPress shouldn’t force the use of a plugin to disable such a frivolous feature (which can’t possibly meet the 80% use requirement). Really hoping I’ve misinterpreted this, so I’m very much looking forward to your reply — cheers!

            • Gary Pendergast 1:32 am on April 7, 2015 Permalink

              This is where the difference between smilies and emojis becomes a little more important. The setting you’re referring to is specifically for converting smilies to images. This setting will continue to exist in WordPress 4.2, but will only apply to smilies.

              Emoji are a different case – they’re text, not images. They’re part of the Unicode standard, and all OSes and browsers are working towards supporting them natively. The problem is that the support is not 100% yet, so the emoji image replacement is all about providing a compatibility layer for those users. As native support is rolled out, the compatibility layer will automatically stop loading the image replacement code.

              While I appreciate that your specific usage may not require emoji support, it’s fairly clear from other platforms (including WordPress.com) that emoji are becoming an important part of communication – a simple way to convey a wide range of emotions in a text format. To take an easy example, look at any business page on Facebook – while the business news posts may not use emoji, you’ll struggle to find a comment thread replying to those posts that don’t contain emoji.

              The important part to remember is that, as of WordPress 4.2, all WordPress sites will support emoji. This is entirely a side effect of our improved Unicode support, and it cannot be disabled (in the same way you cannot disable some of the letters in the alphabet). The compatibility layer is focussed on improving the reading experience for your end users. You can disable that if you really want to, but I think the cases for disabling it are fairly narrow.

            • Julie @Niackery 4:05 pm on April 7, 2015 Permalink

              Thanks very much for that answer, Gary! I continued to read up on emoji some more after leaving that comment and suspected I was misunderstanding the issue. Thanks for clearing it all up — I’m assuming/hoping I’m not the only one who may have been confused about this 😉

              I hope you don’t mind my asking one last question — in Facebook or on my phone, I just click on the image of the emoji I want to use (as opposed to typing the symbols for emoticons, as I understand it). So does this mean that the support added to WordPress will also add buttons to the post editor? How would that work for those using the html editor?

              Thanks again for your help!

            • Gary Pendergast 12:37 am on April 8, 2015 Permalink

              Good news! We wrote a codex page, just to let you know how to use emoji:


              (Summary: On your mobile device, your emoji keyboard will work, both in the WordPress app, and if you visit your Dashboard in your Browser. OS X and Windows 8+ also provide emoji keyboards that you can use.)

            • Julie @Niackery 1:14 pm on April 8, 2015 Permalink

              Oh my goodness! You guys are the best :) Thanks a million!

        • Ihor Vorotnov 2:56 pm on April 5, 2015 Permalink | Log in to Reply

          and, btw, check current page’s url in Chrome on Windows.

          • Gary Pendergast 12:17 am on April 6, 2015 Permalink | Log in to Reply

            Windows 7 and earlier has no native rendering of emoji, so native controls (such as the URL bar) won’t render them correctly. Unfortunately, we don’t have any control over that.

    • marsjaninzmarsa 1:38 am on April 3, 2015 Permalink | Log in to Reply

      Ugh, this article in Feedly (via RSS) looks awful. 😞

    • Primoz Cigler 2:06 am on April 3, 2015 Permalink | Log in to Reply

      Hilarious, never seen the emoji in my URL bar before! 😀

    • Brandon Kraft 2:34 am on April 3, 2015 Permalink | Log in to Reply

      Emoji in URL, while supported per spec and fun, isn’t the greatest experience yet. I have a few notes on it toward the end of http://www.brandonkraft.com/b/2015/03/emoji-wordpress-and-you/

      Now that we support it, I would imagine more folks will sooner than later.

    • Nile Flores 3:14 am on April 3, 2015 Permalink | Log in to Reply

      I’d like to use my own emoji like I did back in the day. I’ve got all these emoticon sets I made that I can choose from. Possible to do this like I could in the past??? For example b2 grins by Alex King became wp grins and you could do this for both post types and comments, and then with a a filter, change the path to the directory of the emoticons you want to use if you wanted to use your own.

      What about like the old days when the comment form on posts, you saw the emoji load and just clicked to add them into your comment?

      • Gary Pendergast 3:40 am on April 3, 2015 Permalink | Log in to Reply

        You can absolutely use a different emoji set!

        As this is based on Twemoji, it uses their naming scheme for the image files. For single character emoji, that looks like “1f4a9.png” (💩). For two character emoji, that looks like “1f1ec-1f1e7.png” (🇬🇧). EmojiOne, for example, uses the same naming scheme, so a plugin to convert emoji to EmojiOne would simple use the “emoji_url” filter to change the location of the emoji images.

        We won’t be doing an emoji selector in core, though. It would be about 400KB of JSON, plus the images – pretty weighty to include on every page. :-)

    • Dave McHale 1:00 pm on April 3, 2015 Permalink | Log in to Reply

      As a chrome user, this does little to excite me. Rectangles in the URL, yay? My email notification about this post was LITTERED with rectangles in Gmail, though most emoji seem to render fine on the page itself. It seems like there are too many rendering support issues for this still… My $0.02

      • Pascal Birchler 5:37 pm on April 3, 2015 Permalink | Log in to Reply

        I use Chrome as well and the emoji in the URL is displayed just fine. And I’m sure Gmail supports emojis just fine too and the rectangles in the email were caused by the Jetpack email subscription.

      • Gary Pendergast 10:37 pm on April 3, 2015 Permalink | Log in to Reply

        Unfortunately, we can’t do anything about Chrome’s URL bar – that only works where the OS has native support for emoji.

        The email problem will be fixed next week: Jetpack’s email subscriptions go through WordPress.com, which I haven’t rolled full emoji support out to, yet. It’s on my list. 😎

    • Chris Van Patten 10:18 am on April 5, 2015 Permalink | Log in to Reply

      I noticed that in Chrome 42.0.2311.68 beta on OS X, the script still kicks in and replaces the emoji even though this version of Chrome fully supports emoji. There’s a noticeable flash as the system-standard emoji get replaced with the Twemoji variants.

      Is that intentional? I thought the aim was for the script to detect if the browser already supports emoji and only kick in if it doesn’t.

      • Gary Pendergast 10:24 am on April 5, 2015 Permalink | Log in to Reply

        This is intentional. Chrome OS X doesn’t render colour emoji if the font weight of the element they’re in is greater than 500 (ie, anything bolder than normal). So, rather than have invisible emoji under some circumstances, we’re replacing all emoji on the page. Once Chrome releases a version that does render emoji in bold elements, the script will automatically stop loading Twemoji.

        For reference, here’s the Chrome ticket for this bug: https://code.google.com/p/chromium/issues/detail?id=441946

  • Gary Pendergast 2:25 am on April 2, 2015 Permalink |
    Tags: , , utf8mb4,   

    The utf8mb4 Upgrade 

    In WordPress 4.2, we’re upgrading tables to utf8mb4, when we can. Your site will only upgrade when the following conditions are met:

    • You’re currently using the utf8 character set.
    • Your MySQL server is version 5.5.3 or higher (including all 10.x versions of MariaDB).
    • Your MySQL client libraries are version 5.5.3 or higher. If you’re using mysqlnd, 5.0.9 or higher.

    The difference between utf8 and utf8mb4 is that the former can only store 3 byte characters, while the latter can store 4 byte characters. In Unicode terms, utf8 can only store characters in the Basic Multilingual Plane, while utf8mb4 can store any Unicode character. This greatly expands the language usability of WordPress, especially in countries that use Han character sets. Unicode isn’t without its problems, but it’s the best option available.

    utf8mb4 is 100% backwards compatible with utf8.

    Due to index size restrictions in MySQL, this does mean we need to re-create a handful of indexes to fit within MySQL’s rules. Using a standard configuration, MySQL allows 767 bytes per index, which for utf8 means 767 bytes / 3 bytes = 255 characters. For utf8mb4, that means 767 bytes / 4 bytes = 191 characters. The indexes that will be resized are:


    And from Multisite:


    Of course, the Multisite (and wp_usermeta) keys obey the DO_NOT_UPGRADE_GLOBAL_TABLES setting. The upgrade will only be attempted once, though we’ll probably add a check in a future WordPress version to see if we can upgrade now (say, if you’ve upgraded your MySQL server since upgrading to WordPress 4.2).

    If you’re a plugin developer and your plugin includes custom tables, please test that your indexes fit within MySQL’s limits. MySQL won’t always produce an error when the index is too big, so you’ll need to manually check the size of each index, instead of relying on automated testing.

    EDIT: One more thing…

    If you’d like to upgrade your custom tables to utf8mb4 (and your indexes are all in order), you can do it really easily with the shiny new maybe_convert_table_to_utf8mb4( $tablename ) function. It’s available in `wp-admin/includes/upgrade.php`, and will sanity check that your tables are entirely utf8 before upgrading.

    • Thomas Townsend 2:33 am on April 2, 2015 Permalink | Log in to Reply

      Hi Gary, glad to see you and the team are looking after the devs too ? I owe you a beer or tow if you even get down to Tampa Florid area…

    • Pippin Williamson 2:36 am on April 2, 2015 Permalink | Log in to Reply

      Excellent, thanks for the notice!

    • Todd Lahman 3:47 am on April 2, 2015 Permalink | Log in to Reply

      Great to see WP Keeps moving forward. Thanks for the update Gary.

    • J.D. Grimes 1:03 pm on April 2, 2015 Permalink | Log in to Reply

      @pento I’ve noticed that the difference in index length can cause notices when updating tables using `dpDelta()`.

      I was getting this error locally, when running the install tests for a plugin against WordPress trunk:

      WordPress database error: [Duplicate key name ‘meta_key’]
      ALTER TABLE wptests_dbdelta_test3 ADD KEY meta_key (meta_key)

      After some debugging, I found that dbDelta() was being thrown off by KEYs for VARCHAR columns, because the description of the key from the database would be like this:

      `KEY meta_key (meta_key(191))`

      But in my CREATE TABLE string, they are defined like this:

      `KEY meta_key (meta_key)`

      Since these don’t match `dbDelta()` attempts to add the key again, resulting in the error. This would happen to users on sites that support utf8mb4, if the plugin is deactivated and then reactivated.

      Kind of related: #31388.

      • Gary Pendergast 1:29 pm on April 2, 2015 Permalink | Log in to Reply

        Thanks for the bug report, @jdgrimes! I’ve recorded it in #31869, I’ll have a think about the best way to fix it.

        In the mean time, a valid work around is to specify the index length in your CREATE TABLE statement – this is what we’ve done in core, hence why I haven’t run into this bug previously.

    • pix365 3:05 pm on April 13, 2015 Permalink | Log in to Reply

      For folks on v4.1.1 ior earlier – Will there be a plugin released that will perform the the sanity checks to ensure existing sites have all the SQL pre-requisites in place before folks even attempt the upgrade to 4.2. ( or have miss understood the above “Edit on the upgrade.php” or is this check is already built in to 4.1.1)

      • Ipstenu (Mika Epstein) 4:00 pm on April 13, 2015 Permalink | Log in to Reply

        The checks are built in.

        > Your site will only upgrade when the following conditions are met:…

        That’s what’s doing it :) If any condition fails, no DB changes. If you’re not even on the right version of MySQL, it won’t do it either.

  • Gary Pendergast 1:26 pm on February 25, 2015 Permalink |
    Tags: , ,   

    Emoji Chat Agenda, February 25, 2015 

    Here’s the agenda for Wednesday’s Emoji Chat in the #core channel on Slack.

    Time/Date: Immediately after the Dev Chat.

    1. Emoji Helper – On platforms that don’t have an emoji keyboard, should we provide one?
    2. Performance – When we’re falling back to Twemoji on large pages, we have to consider how it will perform. Are there faster ways of replacing the emoji characters with images?
    3. Open Floor – There’s still time to rant about the evils of emoji! Let us know how you really feel.
  • Gary Pendergast 12:26 am on February 13, 2015 Permalink |
    Tags: ,   

    Emoji Chat Meeting Notes, February 12, 2015 

    The full meeting archive is available here.

    1. Why we’re doing this

    So, here’s a bit of back story.

    As of r31349, WordPress partially supports emoji. ~60% of WordPress sites are running MySQL 5.5 or later (so can be upgraded to store emoji), and ~40% of browsers natively support emoji. Emoji are a wildly popular method of communication, so we can expect them to be heavily used as soon an they’re available. The problem is, 60%/40% means a really bad experience for a huge number of our users, who’ll try to use emoji, and fail.

    This is where the emoji feature plugin comes in to play. In order to help the 40% of WordPress sites that can’t be upgraded to store emoji natively, the wp_encode_emoji() function will turn them into HTML entities. Due to the unimaginable joy that character sets brings me, this will only be applied to sites using the utf8 character set, which accounts for the vast majority of WordPress sites – utf8 has been the default character set since r4860.

    To help the 60% of browsers that don’t display emoji natively, we’re using the Twemoji image set as a fallback. This lets us show emoji everywhere, without causing extra load where emoji are already supported, mobile browsers being the important example here.

    Now, there have been some concerns brought up previously that I’d like to address.

    “Is this really appropriate for core?”

    Yes. (Obviously that’s my answer, or we wouldn’t be here.) WordPress is is the business of making communication simple and accessible for all. Tech users everywhere have clearly chosen emoji as a means of communication, so it’s up to us to make sure they can do that within WordPress as easily as possible, or risk being left behind.

    “Should we be concerned about changing the images in the future? Wouldn’t we be altering users’ content?”

    No. By using Twemoji only when we can’t provide native support for emoji, it’s a pretty clear message that while the general appearance of emoji stays the same, the actual sprite used can differ significantly between platforms. (For example, every emoji set except Android uses a left hand for :thumbsup:.) As more browsers add native support for emoji, Twemoji usage will drop, reducing even further any impact we can have on users.

    And so, that brings us to today.

    2. The current state of the plugin

    The plugin is very close to done. The editor plugin needs some attention, which @azaozz will be providing soon. There are a few bugs to discuss, which are mostly around fallback behaviour in browsers that don’t support emoji natively. Apart from that, the basic functionality is pretty much how I would expect it to appear in core. It’s had a brief review from the accessibility team, with only some minor alterations needed. The Twemoji images won’t be included in wordpress.zip, as it’s a total of 3.4MB of images. They’re currently hosted on WP.com’s CDN, but we’re investigating other options for where to host them, probably the W.org CDN. Given that the wp-admin Dashboard also loads things from Google, I have no problem with hosting them on an external CDN. There will naturally be a filter on the URL, to allow local hosting for sites that don’t want to use the CDN.

    One of the major concerns at the moment is that we’re going to be splitting data formats, depending on if the site uses the utf8mb4 or the utf8 character set. utf8mb4 stores emoji natively, while utf8 requires us to HTML encode the emoji characters. In the futures, we’ll look at upgrading sites to utf8mb4 if they’ve upgraded their MySQL since WordPress 4.2, but that leaves the potential for mixed encoding – old posts having HTML encoding, new posts having the native characters. A post will be automatically updated to native upon saving, but do we need to consider upgrade routines, to go through all old posts and convert them?

    Export/import also needs thorough testing, particularly when importing and exporting between sites having different character sets.

    3. Unicode 8.0: the future of emoji

    To talk about the future of emoji, you need to know a little bit of history. At the basic level, emoji are all single characters defined within the Unicode standard. However, they also support modifiers. Modifiers are a second character following the first, which usually causes the two characters to be merged into a single character when rendered. A good example of this is flag emoji.

    The character G is U+1F1EC. The character B is U+1F1E7 (these characters are different to their ASCII equivalent). When used individually, they’ll display as that letter. When combined next to each other, they’ll display as the British flag.

    So, Unicode 8.0 will two interesting things: a set of 37 new emoji, and skin tone modifiers. When a skin tone modifier character is placed after any face or person emoji, the emoji will show with that skin tone. Unicode 8.0 is due to be finalised in August 2015, so we and (Twemoji) will be looking at adding support for these then.

    From a technical perspective, it just means we need to be aware that emoji are not always one character, and the methods for detecting multi-character emoji are about to get more complex.

    We’ll also be able to detect if a browser is able to render the new emoji and skin tones, and fall back to Twemoji if they can’t. I don’t have a timeline for when browsers will support the new emoji, so I think it’d be good for us to get ahead of the curve then.

    utf8mb4 stores anything in the Unicode address space, including unallocated characters, so I don’t expect any problems with storage of new emoji.

    • Ryan Hellyer 8:45 am on February 18, 2015 Permalink | Log in to Reply

      I’m not seeing any reason to host them externally. Why not just bundle an extra 3.4 MB into the zip?

      Even better, perhaps this should just stay as a plugin? Maybe I’m wrong, but this seems a very niche thing to me.

      • John James Jacoby 1:17 pm on February 18, 2015 Permalink | Log in to Reply

        Particularly in the US, I can anecdotally tell you that Emoji has had slow adoption for several years, but it’s becoming less-so with the native inclusion of it in iOS. See my comment below about smilies, which also very easily could have been a plugin (and may have originally been at one point?) but they’re so simple and commonplace now, it’s weird not seeing them rendered as images. :)

      • Gary Pendergast 8:26 pm on February 18, 2015 Permalink | Log in to Reply

        Why not just bundle an extra 3.4 MB into the zip?

        They’re compressed PNGs, so they won’t compress much further. They’d be doubling the size of the zip.

        Maybe I’m wrong, but this seems a very niche thing to me.

        Not so much. Emoji are a very popular method of communication by large groups of the the population. If anything, I’d say it’s an oversight that it’s taken us so long to get to supporting emoji.

    • John James Jacoby 1:15 pm on February 18, 2015 Permalink | Log in to Reply

      Something I don’t see mentioned is the current iteration of smilies.

      • Does Emoji support secretly allow us to nix old-school smilies and cut out an option?
      • Can/should Emoji support be tacked on-top-of smilies and work similarly to Slack and use a bunch of string-replacements?

      Understanding the approach and code behind each is different, they are identical to a typical end-user because – rather conveniently for us – smilies in core have no UI (though there are plugins that implement one.)

      I’m imagining either a new UI option for Emoji similar to smilies in writing settings, or having it use that same one, which led me to wondering how dissimilar the underlying functionality needs to be, or if this is an opportunity to update an old API.

      • Gary Pendergast 8:33 pm on February 18, 2015 Permalink | Log in to Reply

        Does Emoji support secretly allow us to nix old-school smilies and cut out an option?

        If I told you, it wouldn’t be a secret anymore, would it? 😉

        Can/should Emoji support be tacked on-top-of smilies and work similarly to Slack and use a bunch of string-replacements?

        Nope. That’s a pretty niche usage, with native OS and browser support for emoji increasing, I’d prefer to keep that kind of hack out of core. It’d also be a super expensive search/replace to have happen at render time.

        Currently, there’s no plan to change/remove the existing smiley option – emoji will just work if you happen to use them, smilies will continue to be optional (but will be replaced by nicer images if you do use them).

        I would like to deprecate the smiley option at some point, but I haven’t given it enough thought to say when that could/should happen.

    • Paal Joachim Romdahl 11:54 pm on February 18, 2015 Permalink | Log in to Reply

      It seems adding emoji to core will cause some perhaps a lot of frustrations for people:

      As the comment seems to be they can add that feature to core but what about…..-insert feature-

      • Paal Joachim Romdahl 1:24 am on February 20, 2015 Permalink | Log in to Reply

        Another thought….. Call it a Easter Egg. Like in games that have hidden features. Press a combination of keys or go and do something special in a game and it unlocks something cool.
        I think each release of WordPress could include a Easter Egg (“just” for fun and unique use). Emoji is the first Easter Egg and fits perfectly as Easter is around the time of the 4.2 release.

        So when people ask why it was included in WordPress 4.2 one can say it is an easter egg. Meaning it was included “just” for fun (of course it has it purpose for being included).

  • Gary Pendergast 11:25 pm on February 11, 2015 Permalink |
    Tags: ,   

    Emoji Chat Agenda, February 12, 2015 

    Here’s the agenda for Thursday’s Emoji Chat in the #core channel on Slack.

    Time/Date: February 12 2015 23:00 UTC

    1. Why we’re doing this – @pento
    2. The current state of the plugin – @pento
    3. Unicode 8.0, and future plans – @pento
    4. Open Floor/ranting about the evils of emoji – everyone

    See you tomorrow!

  • Gary Pendergast 3:07 am on February 9, 2015 Permalink |
    Tags: ,   

    Emoji Feature Plugin for 4.2 

    It’s time for a weekend fun feature! Now that #21212 is complete, WordPress kind of supports Emoji (for the 60% of WordPress sites using MySQL 5.5+, and the 30-40% (by usage) of browsers that natively display Emoji – including when Chrome for OS X adds support in the next month or so).

    In order to complete this support, I’ve created a feature plugin called x1f4a9, which makes use of Twitter’s Open Source twemoji icon set, the same as WordPress.com recently added.

    I’ve added a few tickets to the Github project, feel free to add any others you think of, and pull requests are always welcome! If you’d like to test the plugin, daily builds are available from the plugin repo.

    (And if you’re using MySQL older than 5.5, please pay special attention to this ticket.)

    • Aaron Brazell 3:20 am on February 9, 2015 Permalink | Log in to Reply

      We can do this but we can’t do Postgres support? We can do this but dynamic image sizes is plugin territory?

      • Gary Pendergast 3:59 am on February 9, 2015 Permalink | Log in to Reply

        That’s correct! :-)

        Postgres has very low usage – the plugin has < 1000 installs. The massive amount of rearchitecture it would take to support it in core is totally not a viable option.

        Dynamic image sizing is a very CPU intensive task, which would make your average shared host fall over. We don't really want to make every host explode, so it stays in plugin territory – easily usable by folks who have the server power, or as a service like Jetpack's Photon.

        And because I've been working on architecture things for the last 6 months or so, I get to choose to work on some fun things every now and then. 😉

        • Aaron Brazell 2:38 pm on February 9, 2015 Permalink | Log in to Reply

          Certainly no one wants to tell you what you can or should work on. I question its’ existence as a featured plugin though. I mean, we could make my Superhero Avatars a featured plugin based on your argument. 😉

          Really, not trying to be argumentative, but this isn’t the progress that I, for one, think WordPress core should be pursuing (and I do understand that featured plugins are not core, but they are meant to be potentially core material down the road).


          • George Stephanis 2:55 pm on February 9, 2015 Permalink | Log in to Reply

            It’s easy to fall into groupthink, but the fact of the matter is that a lot of users want to use Emoji, and Core’s inability to support that is a stumbling block for many users.

            Good software should work out of the box.

      • Helen Hou-Sandi 4:32 pm on February 9, 2015 Permalink | Log in to Reply

        We’re talking about support for characters that are available in a standard and standardized, easily and frequently accessed keyboard palette on a non-insignificant percentage of devices, both desktop and mobile. Also, while yes WordPress is a project and there are overarching directions and initiatives, most things are still worked on by individual contributors. Belief that X should be worked on instead of Y is largely a fallacy, because most plugins and patches are whatever a given contributor is both inspired to and has the expertise to work on. In this case, this is a natural extension of the exhaustive work Gary has done on the architectural support front for the last few months.

      • petermolnar 9:14 am on February 10, 2015 Permalink | Log in to Reply

        You’ve spoken from the heart of many.

    • Chris Van Patten 4:58 pm on February 9, 2015 Permalink | Log in to Reply

      I took a quick look at the feature plugin and like most of it… the wp_encode_emoji function is clever and it all looks nice, but I’m really don’t like that it uses Twemoji. I understand why the Twemoji approach is gaining popularity, but it seems like a waste to hack in the emoji support when browsers are on track to get it implemented themselves. On browsers that do support emoji, this approach causes unnecessary file requests and deviates from the system’s standard emoji set. Then what happens when browsers finally do support it en masse? You can’t really remove the support without opening up to user complaints that the emoji images have changed and inevitably screwed up some aspect of their site’s design.

      I think all the technical fixes in the plugin make sense (encoding the emoji and whatnot) but I really think the Twemoji stuff should either be an opt-in setting or a separate non-core plugin.

      One dude’s opinion, anyway :)

      • Gary Pendergast 12:58 am on February 10, 2015 Permalink | Log in to Reply

        I think there are good arguments either way. As you mentioned, it’s a bit weird to override native sprites, and it does have the potential to cause backwards compatibility issues. There’s a little bit of extra data to be downloaded, sure, but I think it’s worth the tradeoff. In the same way that most sites load jQuery, it’s a bit more data, but provides a much better experience for everyone involved.

        I think it’s important to look towards provide a consistent experience across all WordPress sites. WordPress being as big as it is, I think we qualify as our own ecosystem, we can look to provide a smoother experience, regardless of platform.

        I also think we’ll run into problems with inconsistent emoji support in different platforms that current support emoji. Unicode 8 (due to be finalised later this year) defines a bunch of new emoji, which are currently not supported on any platform. It also defines skin tone modifiers for existing emoji, which are trickier to implement, so will potentially delay the rollout on various platforms. Depending on how long it takes to draw the new images, we could be rolling out Unicode 8 support before anyone else.

        Finally, we have an existing relationship with the Twemoji project, through their partnership with WordPress.com. They’ve been great people to work with on that project, I’m optimistic that WordPress core can have a great relationship with them, too. :-)

        • Chris Van Patten 6:19 pm on February 10, 2015 Permalink | Log in to Reply

          I don’t want to come across as being altogether anti-Twemoji as I think it’s a really interesting approach and clearly it’s well done for what it is, but I think it’s worth bearing in mind that it’s a bit of a different case for WordPress.com and Twitter vs self-hosted WordPress.

          It would be easy for Twitter (for instance) to decide that browser support is Good Enough™ and subsequently disable Twemoji on their platform, because they fully control the user experience and can verify it works. Same for WordPress.com (to a lesser extent)—the themes are tightly controlled and it’d be trivial to write a regression test before disabling Twemoji to verify it won’t break things. For self-hosted WordPress, there are so many other moving pieces, and really once you turn this on you kinda can’t turn it off. Even if all major browsers fully support emoji in all its forms within a year or so (optimistic, sure; but I do think we’ll see really high adoption rates of most emoji features within a year), you’re kinda stuck, because who knows what might break for the users if you turn it off.

          I guess ultimately I’m just really uncomfortable because it just feels like a hack. Emoji are supposed to be *fonts*, not images. That’s by design. Really, the whole image replacement technique reminds me a lot of the old SIFr type replacement system, that replaced text with a Flash embed so you could use custom fonts. Sure it was more consistent, but it was also slow and annoying. But it was also okay because it was something only devs implemented, and it was easy for the dev to switch to web fonts once they received wider browser support.

          Here, it’s making that decision on behalf of users who might not fully understand the implications.

          Just to close things off I really don’t want to come across as negative and anti-emoji and whatever. I just think the Twemoji approach might not be the best—or at least not as a platform-wide forced-on solution.

          (I’d also say that in regards to the additional HTTP requests, many themes and plugins are already *really bad* about this; we’ve all seen the random WP site with 30 plugins loading dozens of JS dependencies and the poorly developed themes with never-ending tags. With that in mind, I really think adding additional platform-wide HTTP requests—for emoji or anything else—should be a really deeply and seriously considered issue. We’re pretty lucky in the developed world to have relatively fast Internet where additional requests aren’t that noticeable, but for users in the developing world, or users in rural areas without access to reliable broadband, those requests add up fast. This is goes for mobile users across the world as well. It already sucks waiting for web fonts to load on a 3G network… why add emoji to that too when most modern mobile browsers support emoji natively?)

          • Gary Pendergast 10:59 pm on February 10, 2015 Permalink | Log in to Reply

            That’s a good argument, I absolutely agree that slow connections and mobile rendering should be taken into account.

            There seems to be a way to detect if emoji is supported in a browser or not (rather than maintaining a list of browsers that support emoji). I’ve added a ticket to track it here:


      • Gary Pendergast 12:59 am on February 10, 2015 Permalink | Log in to Reply

        Oh, and thank you for noticing wp_encode_emoji(). I’m just a little bit proud of that bit of code. 😀

    • Lara Littlefield 9:25 pm on February 9, 2015 Permalink | Log in to Reply


    • Michael Arestad 6:16 pm on February 10, 2015 Permalink | Log in to Reply

      As much as I love emoji, I don’t know that they should be in core. The reason is that once these make it into core, they will stay in core forever because they would be considered user content. This means, that should we update this set ever, there would need to be controls for versions of sets (as well as bundling these images forever) or we’d be messing with users’ content.

      • Gary Pendergast 11:01 pm on February 10, 2015 Permalink | Log in to Reply

        Assuming we only load Twemoji on browsers that don’t natively support emoji, I think that would do a good job of setting expectations that emoji look different on different browsers. That makes their appearance trimming, rather than content.

      • BizzThemes 10:31 am on February 11, 2015 Permalink | Log in to Reply

        Good point. Same should apply to Press This. WordPress is not a social network and doesn’t need these social networking tools.

        • John Blackbourn 9:45 pm on February 11, 2015 Permalink | Log in to Reply

          What’s this got to do with social networking?

          • BizzThemes 8:28 am on February 13, 2015 Permalink | Log in to Reply

            Press This is a sharing feature, no? FB has it, TW has it, but not sure why WP.org needs this in core. Sharing tools work great on social networks, where content consumption is organized around sharing stuff from the web. But WP.org is not a social network. We should also support original content. Imagine, who would read Medium if there were re-posted articles on it.

    • Piet Bos 7:44 am on March 25, 2015 Permalink | Log in to Reply

      I won’t go into whether or not I like this. My problem with the emoji feature is that it adds javascript to the footer of my site. Apart from the fact that I don’t want to add unnecessary calls for things I don’t use, one of the scripts is calling the URL http://s0.wp.com/. This URL is blocked in China where I currently reside, which means that my own site now takes forever to load, because it first has to wait until the call to http://s0.wp.com/ times out.

      This cannot be the meaning of progress…

      Is there a way to turn this mess off for people that don’t want it, or do those people have to jump through all kinds of hoops to disable the lot?

    • Piet Bos 8:08 am on March 25, 2015 Permalink | Log in to Reply

      Fortunately I found a way to disable the lot. Can’t add it to my earlier comment as that is still under moderation, but for anyone not interested in this “feature” and/or lives in a location where s0.wp.com is blocked, have a look at https://wordpress.org/plugins/disable-emojis/

    • Rene Hermenau 4:11 pm on April 13, 2015 Permalink | Log in to Reply

      Or better use the one by otto: https://wordpress.org/plugins/classic-smilies/

      I stumbled over this thread when i was looking for a way to disable this new feature as i dont need any emoj handling for my pages. Could you please give user the possibility to disable this feature completely?

      Imho it should be part of the WordPress codex:
      New core functions which are not essential must have a disable function.


      Its not much kind amusement to have the need for another plugin only to disable a non-essential “fun-feature”;-)

  • Gary Pendergast 6:20 am on April 7, 2014 Permalink
    Tags: , , , ,   

    MySQL in WordPress 3.9 

    In WordPress 3.9, we added an extra layer to WPDB, causing it to switch to using the mysqli PHP library, when using PHP 5.5 or higher.

    For plugin developers, this means that you absolutely shouldn’t be using PHP’s mysql_*() functions any more – you can use the equivalent WPDB functions instead.


    There are a few different options for replacing the query functions, depending on what you want to do:

    As a drop in replacement to run a query that you don’t expect a return value from (i.e., an INSERT, UPDATE or DELETE query), use $wpdb->query(). This will always return the number of rows effected by the query.

    Alternatively, $wpdb->insert(), $wpdb->update(), $wpdb->delete() and $wpdb->replace() are all helper functions that will automatically escape your data, then generate and run the queries for you. Ideally, you should never need to write an SQL statement!


    If you have a SELECT query, for which you’d normally do a mysql_query() followed by a mysql_fetch_*(), WPDB lets you combine this into one function call.

    To get all of the results from a query that returns more than one row, use $wpdb->get_results() to return an array of objects containing your data.

    There are also some shortcut functions for common usage:

    If you only need a single row from your query, $wpdb->get_row() will return just the data object from that row.

    If you only need a single column from a single row, $wpdb->get_var() will return only that field.

    And if you need a single column, $wpdb->get_col() will return an array of all the data from that column.


    For a drop in replacement, you can use esc_sql(). That said, we strongly recommend switching to $wpdb->prepare(), instead. We have a pretty thorough tutorial available for $wpdb->prepare().


    If you need to get the Insert ID from the last query, $wpdb->insert_id is where you need to look.

    Updating your plugin to use WPDB will also future proof it for if we make changes to how WordPress connects to the database – we’ll always maintain backwards compatibility with the current WPDB interface.

    For more reading, check the WPDB Codex page, and #21663.

    If you’re using MySQL in a way that I haven’t covered here, please post it in the comments, we’d be happy to help you out!

    • Samuel Wood (Otto) 6:23 am on April 7, 2014 Permalink | Log in to Reply

      Note: $wpdb->escape() is deprecated. Please use esc_sql() instead. Or $wpdb->prepare(), of course.

      Also note that $wpdb->escape() is not a proper replacement for mysql_real_escape_string() to begin with, as it only does weak escaping.

    • Doug Wollison 12:09 pm on April 7, 2014 Permalink | Log in to Reply

      I though I heard it was going to switch to PDO, not MySQLi?

      • Gary Pendergast 12:30 pm on April 7, 2014 Permalink | Log in to Reply

        That’s still on the cards for a future release, but it will be a significantly bigger project. The primary goal with this change was to stop using ext/mysql on PHP 5.5+, where it’s deprecated.

    • Brian Layman 4:10 pm on April 7, 2014 Permalink | Log in to Reply

      Wow! Huge change, and with the MySQL extension being deprecated in PHP 5.5 it’s a good one. I am trying to remember if 5.5 REQUIRES MySQLi to be built in. Does anyone know? Should you also throw in a check for the existence of mysqli_connect?

      Also, are there plans to start preparing queries? (Not WP prepare but the database meaning of prepare.)

      • Gary Pendergast 3:06 am on April 10, 2014 Permalink | Log in to Reply

        We do check that mysqli_connect() exists before trying to use it. (See wpdb::__construct().)

        There are no plans for adding statement prepare in the near future, though it is something I would like to get to!

    • webaware 10:46 pm on April 7, 2014 Permalink | Log in to Reply

      Plugin writers sometimes call mysql_get_server_info() to get the raw version information that $wpdb->db_version() strips out. Since the database handle isn’t public (nor is the mysqli indicator), there isn’t a way to do this in 3.9-beta3. I’ve opened a trac in hopes we can get a new method added to class wpdb so that plugin writers can still access this raw version information:


      • Gary Pendergast 5:28 am on April 8, 2014 Permalink | Log in to Reply

        To summarise the ticket, for anyone using mysql_get_server_info() – the best option is to have a switch in your code for mysqli, we’ll look at expanding $wpdb->db_version() at a later date.

        if ( empty( $wpdb->use_mysqli ) ) {
        	$ver = mysql_get_server_info();
        } else {
        	$ver = mysqli_get_server_info( $wpdb->dbh );
    • Claudio 8:51 pm on April 13, 2014 Permalink | Log in to Reply

      I use mysql_connect to test DB connectivity. How should I replace it for WP3.9/PHP5.5 compatibility?

      • Gary Pendergast 12:13 am on April 14, 2014 Permalink | Log in to Reply

        To test the connection, you can use $wpdb->check_connection(), which will check that the connection is up, and try to reconnect if it isn’t.

        This is particularly useful for long running cron jobs, where the MySQL connection might drop out due to inactivity, but there’s no actual problem with the server.

    • Ross Seddon 6:08 am on May 5, 2014 Permalink | Log in to Reply

      My web site which was designed using a standard off shelf theme I’m advised by our web site managers they site is providing the response “Access denied for user ‘www-data’@’localhost’ (using password: NO).” They advise
      Quote: “the one statement in there is exactly the issue: For plugin developers, this means that you absolutely shouldn’t be using PHP’s mysql_*() functions any more – you can use the equivalent WPDB functions instead.”

      The developer how wrote the plugin that is used site wide on your site for the skin, uses this no longer available method. All there work needs to be updated to use the correct method of connecting to the database. We know what is required, just that the plugin / skin is none of our work, and it is time consuming to fix.”

      We been down now for 3 weeks in terms of accessing site. Is what your referring to this thread relative to my site problem site http://www.totallyoutdoors.com.au

      how time consuming is this problem?


    • lwall 1:07 pm on May 15, 2014 Permalink | Log in to Reply

      I am having troubles with admin permissions at different points.

      When clicking on “Posts”, I get error “Invalid post type”, or when trying to create a new post, there is no save box/button.

      It also happens when trying to change options in one of my themes in other places with plugins. I get “You do not have sufficient permissions to access this page.”

      For the most part, the back-end is functional, the front-end is fully functional.

      I am using appengine 1.9.3 and wordpress 3.9, python 2.7.6.

      I have uninstalled 1.9.3 and updated to 1.9.4, I have also accepted WordPress’ request to install 3.9.1, the problem persists.

      I have installed the 1.9.3, 3.9, 2.7.6 configuration on a different machine where appengine was never installed before and the same problem occurs.

      I am discarding plugins because there were no plugins in the separate machine.

      I had appengine 1.9.0 and 1.9.3 working with WordPress 3.8.1. The problem started a few days ago after a number of upgrades (from 1.9.3 to 1.9.4, and into wordpress 3.9.1).

      Could this extra layer to WPDB be the source of this?

    • Michael Simpson 1:49 pm on July 11, 2014 Permalink | Log in to Reply

      For a plugin, I would like to be able to do unbuffered queries (MYSQLI_USE_RESULT). This is to handle cases when there are a lot of rows being returned and I want to keep the memory footprint down. wpdb doesn’t support this well and it would be nice if it would.

      For now, I have to create a new wpdb object and directly call mysqli_query($wpdb->dbh, $sql, MYSQLI_USE_RESULT); It is not using MySQLi, then I need to call mysql_unbuffered_query($sql, $wpdb->dbh);

      To know which one to call, I would like to consult $wpdb->use_mysqli but I cannot because it is private and there is no accessor method. So I have to copy the code from wp-db.php to determine it.

      For a start, it would be nice to be able to access $wpdb->use_mysqli. Even better would be an API on $wpdb to do unbuffered queries.


      • Gary Pendergast 2:02 pm on July 11, 2014 Permalink | Log in to Reply

        You can access $wpdb->use_mysqli directly, because of the wpdb::__get() magic getter.

        There’s unlikely to ever be an API in WPDB for doing unbuffered queries. There are too many gotchas that will just cause maintenance problems.

compose new post
next post/next comment
previous post/previous comment
show/hide comments
go to top
go to login
show/hide help
shift + esc
Skip to toolbar