Make WordPress Core

Updates from Gary Pendergast Toggle Comment Threads | Keyboard Shortcuts

  • Gary Pendergast 11:17 pm on April 2, 2015 Permalink
    Tags: , , ,   

    OMG EMOJI 😎 

    One of the fun bits of the utf8mb4 upgrade is that we can now store emoji! Once your site is upgraded to utf8mb4, it can natively store any emoji character. There were a couple of problems with that, though:

    • Some browsers don’t know how to render emoji 👎, or they have bugs in their implementation 😢. Notably, Chrome either doesn’t work or has bugs, older versions of IE don’t work, and Firefox has bugs.
    • Not all sites will be able to upgrade to utf8mb4, which means they’ll be unable to save emoji characters that they enter.

    Not being able to use emoji makes everyone a sad panda (😭🐼), so we need to fix this. There are a few moving parts of our emoji support, so lets go through them.

    utf8 backwards compatibility

    If a site can’t be upgraded to utf8mb4, we convert emoji to their HTML-encoded equivalent, and store that, instead. From a UI perspective, post editing works as expected 🎉. Because fields need to be white listed to support this, we don’t allow it everywhere – just post_title, post_content and post_excerpt. We also allow it in the site title and the site description.

    Browser support

    There’s a small compatibility check included on every page, both on the front end, and in the Dashboard. For those interested, this adds 1-4ms (⚡️-fast!) to the page render time – the aim was to keep this as low as possible, to avoid affecting your UX. If you notice a little chunk of compressed JS at the top your HTML, that’s probably it. If you’d like to check out how it works in a more readable format, have a look through wp-emoji-loader.js.

    Email ✉️ and RSS 📯 (There’s no RSS emoji, give me a break.)

    Of course, the JS shim won’t work in email and RSS. So, we replace all emoji with a static PNG version in those cases.

    TinyMCE 📝

    In addition to the browser support JS, there’s a TinyMCE plugin that handles keeping emoji looking good, while you type. It’s pretty magical.

    Taxonomies and URL slugs

    You can totally make taxonomies and URL slugs with emoji in them. Because we love you, and want you to be happy. 😀

    So, that’s about it. If you have any questions about the implementation, drop them in the comments below.


    • Scott Kingsley Clark 11:19 pm on April 2, 2015 Permalink | Log in to Reply

      Does this mean post type names / taxonomy names / option names / meta keys support emoji now?

    • Adam Silverstein 11:24 pm on April 2, 2015 Permalink | Log in to Reply


    • Lara Littlefield 11:38 pm on April 2, 2015 Permalink | Log in to Reply


    • Mustafa Uysal 11:41 pm on April 2, 2015 Permalink | Log in to Reply


    • Stephen Edgar 11:56 pm on April 2, 2015 Permalink | Log in to Reply


    • bravokeyl 12:11 am on April 3, 2015 Permalink | Log in to Reply

      Emoji in URL’s , Cool 1F382

    • Ihor Vorotnov 12:30 am on April 3, 2015 Permalink | Log in to Reply

      omg, will there be a way to prevent upgrading and completely remove this compatibility check and that ‘little chunk of compressed javascript’ which is a thing Google does not tolerate?

      • Gary Pendergast 12:36 am on April 3, 2015 Permalink | Log in to Reply

        There’s a plugin to disable emoji compatibility. It’s currently broken (it’s unhooking the wrong filters), but I assume it’ll be fixed before 4.2 is released.

        which is a thing Google does not tolerate

        Can you please clarify what you mean by that?

        • Ihor Vorotnov 2:53 pm on April 5, 2015 Permalink | Log in to Reply

          Google’s recommendations for faster and better sites include “remove any inline styles and scripts from your html source”. Which is actually a common sense – scripts are scripts, markup is markup. We were fighting for years to remove inline WP gallery styles and now we’ve got inline scripts. Awesome)

          It’s a good thing to include this compatibility in core, but turning emoji support (and adding extra inline script) by default isn’t. Most sites (not blogs) have serious tone of voice and don’t need emoji. Those who want them can turn them on in options or using add_theme_support( ’emoji’ ) for example. That’s how it should be, IMHO, not vice versa.

          • Gary Pendergast 12:16 am on April 6, 2015 Permalink | Log in to Reply

            Google’s recommendations for faster and better sites include “remove any inline styles and scripts from your html source”.

            That is true for large scripts and styles, but not for small scripts.

            For scripts needed for page rendering, Google recommends putting them inline, to avoid extra network requests:

            Scripts that are necessary to render page content can be inlined to avoid extra network requests, however the inlined content needs to be small and must execute quickly to deliver good performance.


            • Julie @Niackery 2:32 pm on April 6, 2015 Permalink

              Hi Gary, thanks very much for answering all the questions here! I’m finding the info very useful. I’m just wondering about the second part of the comment above. Currently my settings are not to convert emoji — so as of 4.2 this setting won’t exist anymore? Emoji will be converted automatically regardless of settings? Or will I have to readjust my settings? I really despise the idea of installing a plugin to configure something I should be able to do in my settings. I sincerely hope WordPress isn’t forcing emoji on every single install. As stated above, a great many sites will not appreciate having emoji turned on automatically, especially without the ability to disable without a plugin. WordPress shouldn’t force the use of a plugin to disable such a frivolous feature (which can’t possibly meet the 80% use requirement). Really hoping I’ve misinterpreted this, so I’m very much looking forward to your reply — cheers!

            • Gary Pendergast 1:32 am on April 7, 2015 Permalink

              This is where the difference between smilies and emojis becomes a little more important. The setting you’re referring to is specifically for converting smilies to images. This setting will continue to exist in WordPress 4.2, but will only apply to smilies.

              Emoji are a different case – they’re text, not images. They’re part of the Unicode standard, and all OSes and browsers are working towards supporting them natively. The problem is that the support is not 100% yet, so the emoji image replacement is all about providing a compatibility layer for those users. As native support is rolled out, the compatibility layer will automatically stop loading the image replacement code.

              While I appreciate that your specific usage may not require emoji support, it’s fairly clear from other platforms (including WordPress.com) that emoji are becoming an important part of communication – a simple way to convey a wide range of emotions in a text format. To take an easy example, look at any business page on Facebook – while the business news posts may not use emoji, you’ll struggle to find a comment thread replying to those posts that don’t contain emoji.

              The important part to remember is that, as of WordPress 4.2, all WordPress sites will support emoji. This is entirely a side effect of our improved Unicode support, and it cannot be disabled (in the same way you cannot disable some of the letters in the alphabet). The compatibility layer is focussed on improving the reading experience for your end users. You can disable that if you really want to, but I think the cases for disabling it are fairly narrow.

            • Julie @Niackery 4:05 pm on April 7, 2015 Permalink

              Thanks very much for that answer, Gary! I continued to read up on emoji some more after leaving that comment and suspected I was misunderstanding the issue. Thanks for clearing it all up — I’m assuming/hoping I’m not the only one who may have been confused about this 😉

              I hope you don’t mind my asking one last question — in Facebook or on my phone, I just click on the image of the emoji I want to use (as opposed to typing the symbols for emoticons, as I understand it). So does this mean that the support added to WordPress will also add buttons to the post editor? How would that work for those using the html editor?

              Thanks again for your help!

            • Gary Pendergast 12:37 am on April 8, 2015 Permalink

              Good news! We wrote a codex page, just to let you know how to use emoji:


              (Summary: On your mobile device, your emoji keyboard will work, both in the WordPress app, and if you visit your Dashboard in your Browser. OS X and Windows 8+ also provide emoji keyboards that you can use.)

            • Julie @Niackery 1:14 pm on April 8, 2015 Permalink

              Oh my goodness! You guys are the best :) Thanks a million!

        • Ihor Vorotnov 2:56 pm on April 5, 2015 Permalink | Log in to Reply

          and, btw, check current page’s url in Chrome on Windows.

          • Gary Pendergast 12:17 am on April 6, 2015 Permalink | Log in to Reply

            Windows 7 and earlier has no native rendering of emoji, so native controls (such as the URL bar) won’t render them correctly. Unfortunately, we don’t have any control over that.

    • marsjaninzmarsa 1:38 am on April 3, 2015 Permalink | Log in to Reply

      Ugh, this article in Feedly (via RSS) looks awful. 😞

    • Primoz Cigler 2:06 am on April 3, 2015 Permalink | Log in to Reply

      Hilarious, never seen the emoji in my URL bar before! 😀

    • Brandon Kraft 2:34 am on April 3, 2015 Permalink | Log in to Reply

      Emoji in URL, while supported per spec and fun, isn’t the greatest experience yet. I have a few notes on it toward the end of http://www.brandonkraft.com/b/2015/03/emoji-wordpress-and-you/

      Now that we support it, I would imagine more folks will sooner than later.

    • Nile Flores 3:14 am on April 3, 2015 Permalink | Log in to Reply

      I’d like to use my own emoji like I did back in the day. I’ve got all these emoticon sets I made that I can choose from. Possible to do this like I could in the past??? For example b2 grins by Alex King became wp grins and you could do this for both post types and comments, and then with a a filter, change the path to the directory of the emoticons you want to use if you wanted to use your own.

      What about like the old days when the comment form on posts, you saw the emoji load and just clicked to add them into your comment?

      • Gary Pendergast 3:40 am on April 3, 2015 Permalink | Log in to Reply

        You can absolutely use a different emoji set!

        As this is based on Twemoji, it uses their naming scheme for the image files. For single character emoji, that looks like “1f4a9.png” (💩). For two character emoji, that looks like “1f1ec-1f1e7.png” (🇬🇧). EmojiOne, for example, uses the same naming scheme, so a plugin to convert emoji to EmojiOne would simple use the “emoji_url” filter to change the location of the emoji images.

        We won’t be doing an emoji selector in core, though. It would be about 400KB of JSON, plus the images – pretty weighty to include on every page. :-)

        • Nile Flores 11:06 am on April 3, 2015 Permalink | Log in to Reply

          Yay! That makes me a happy panda… lol

        • marsjaninzmarsa 9:33 pm on May 1, 2015 Permalink | Log in to Reply

          As this is based on Twemoji, it uses their naming scheme for the image files. For single character emoji, that looks like “1f4a9.png” (💩). For two character emoji, that looks like “1f1ec-1f1e7.png” (🇬🇧). EmojiOne, for example, uses the same naming scheme, so a plugin to convert emoji to EmojiOne would simple use the “emoji_url” filter to change the location of the emoji images.

          You’ve inspired me. 😀

    • Dave McHale 1:00 pm on April 3, 2015 Permalink | Log in to Reply

      As a chrome user, this does little to excite me. Rectangles in the URL, yay? My email notification about this post was LITTERED with rectangles in Gmail, though most emoji seem to render fine on the page itself. It seems like there are too many rendering support issues for this still… My $0.02

      • Pascal Birchler 5:37 pm on April 3, 2015 Permalink | Log in to Reply

        I use Chrome as well and the emoji in the URL is displayed just fine. And I’m sure Gmail supports emojis just fine too and the rectangles in the email were caused by the Jetpack email subscription.

      • Gary Pendergast 10:37 pm on April 3, 2015 Permalink | Log in to Reply

        Unfortunately, we can’t do anything about Chrome’s URL bar – that only works where the OS has native support for emoji.

        The email problem will be fixed next week: Jetpack’s email subscriptions go through WordPress.com, which I haven’t rolled full emoji support out to, yet. It’s on my list. 😎

    • Chris Van Patten 10:18 am on April 5, 2015 Permalink | Log in to Reply

      I noticed that in Chrome 42.0.2311.68 beta on OS X, the script still kicks in and replaces the emoji even though this version of Chrome fully supports emoji. There’s a noticeable flash as the system-standard emoji get replaced with the Twemoji variants.

      Is that intentional? I thought the aim was for the script to detect if the browser already supports emoji and only kick in if it doesn’t.

      • Gary Pendergast 10:24 am on April 5, 2015 Permalink | Log in to Reply

        This is intentional. Chrome OS X doesn’t render colour emoji if the font weight of the element they’re in is greater than 500 (ie, anything bolder than normal). So, rather than have invisible emoji under some circumstances, we’re replacing all emoji on the page. Once Chrome releases a version that does render emoji in bold elements, the script will automatically stop loading Twemoji.

        For reference, here’s the Chrome ticket for this bug: https://code.google.com/p/chromium/issues/detail?id=441946

    • Mike Crantea 5:23 pm on April 23, 2015 Permalink | Log in to Reply

      I noticed that when changing the option in the admin from Writing to not convert emoticons to graphics, the inline styles and the emoji JS still comes through. I believe they should be ignored when the site owner does not want to use this feature.

      • Ipstenu (Mika Epstein) 11:58 am on April 24, 2015 Permalink | Log in to Reply

        Comes through on the editor or on the pages people visit?

        Unlike smiles, one can enter emoji into text without using emoticons like :)

        This, for example, is emoji — 😐

        This is a smilie: 😀

        So one can have both, one, or none

        • Monika 10:44 am on April 25, 2015 Permalink | Log in to Reply

          “Comes through on the editor or on the pages people visit?”
          The scripts are still at wp_head, if I disable “convert smilies” or not, this is frustrating.
          And my old smilies are broken. 😥 and so on… :(

  • Gary Pendergast 2:25 am on April 2, 2015 Permalink
    Tags: , , utf8mb4,   

    The utf8mb4 Upgrade 

    In WordPress 4.2, we’re upgrading tables to utf8mb4, when we can. Your site will only upgrade when the following conditions are met:

    • You’re currently using the utf8 character set.
    • Your MySQL server is version 5.5.3 or higher (including all 10.x versions of MariaDB).
    • Your MySQL client libraries are version 5.5.3 or higher. If you’re using mysqlnd, 5.0.9 or higher.

    The difference between utf8 and utf8mb4 is that the former can only store 3 byte characters, while the latter can store 4 byte characters. In Unicode terms, utf8 can only store characters in the Basic Multilingual Plane, while utf8mb4 can store any Unicode character. This greatly expands the language usability of WordPress, especially in countries that use Han character sets. Unicode isn’t without its problems, but it’s the best option available.

    utf8mb4 is 100% backwards compatible with utf8.

    Due to index size restrictions in MySQL, this does mean we need to re-create a handful of indexes to fit within MySQL’s rules. Using a standard configuration, MySQL allows 767 bytes per index, which for utf8 means 767 bytes / 3 bytes = 255 characters. For utf8mb4, that means 767 bytes / 4 bytes = 191 characters. The indexes that will be resized are:


    And from Multisite:


    Of course, the Multisite (and wp_usermeta) keys obey the DO_NOT_UPGRADE_GLOBAL_TABLES setting. The upgrade will only be attempted once, though we’ll probably add a check in a future WordPress version to see if we can upgrade now (say, if you’ve upgraded your MySQL server since upgrading to WordPress 4.2).

    If you’re a plugin developer and your plugin includes custom tables, please test that your indexes fit within MySQL’s limits. MySQL won’t always produce an error when the index is too big, so you’ll need to manually check the size of each index, instead of relying on automated testing.

    EDIT: One more thing…

    If you’d like to upgrade your custom tables to utf8mb4 (and your indexes are all in order), you can do it really easily with the shiny new maybe_convert_table_to_utf8mb4( $tablename ) function. It’s available in `wp-admin/includes/upgrade.php`, and will sanity check that your tables are entirely utf8 before upgrading.

    • Thomas Townsend 2:33 am on April 2, 2015 Permalink | Log in to Reply

      Hi Gary, glad to see you and the team are looking after the devs too ? I owe you a beer or tow if you even get down to Tampa Florid area…

    • Pippin Williamson 2:36 am on April 2, 2015 Permalink | Log in to Reply

      Excellent, thanks for the notice!

    • Todd Lahman 3:47 am on April 2, 2015 Permalink | Log in to Reply

      Great to see WP Keeps moving forward. Thanks for the update Gary.

    • J.D. Grimes 1:03 pm on April 2, 2015 Permalink | Log in to Reply

      @pento I’ve noticed that the difference in index length can cause notices when updating tables using `dpDelta()`.

      I was getting this error locally, when running the install tests for a plugin against WordPress trunk:

      WordPress database error: [Duplicate key name ‘meta_key’]
      ALTER TABLE wptests_dbdelta_test3 ADD KEY meta_key (meta_key)

      After some debugging, I found that dbDelta() was being thrown off by KEYs for VARCHAR columns, because the description of the key from the database would be like this:

      `KEY meta_key (meta_key(191))`

      But in my CREATE TABLE string, they are defined like this:

      `KEY meta_key (meta_key)`

      Since these don’t match `dbDelta()` attempts to add the key again, resulting in the error. This would happen to users on sites that support utf8mb4, if the plugin is deactivated and then reactivated.

      Kind of related: #31388.

      • Gary Pendergast 1:29 pm on April 2, 2015 Permalink | Log in to Reply

        Thanks for the bug report, @jdgrimes! I’ve recorded it in #31869, I’ll have a think about the best way to fix it.

        In the mean time, a valid work around is to specify the index length in your CREATE TABLE statement – this is what we’ve done in core, hence why I haven’t run into this bug previously.

    • pix365 3:05 pm on April 13, 2015 Permalink | Log in to Reply

      For folks on v4.1.1 ior earlier – Will there be a plugin released that will perform the the sanity checks to ensure existing sites have all the SQL pre-requisites in place before folks even attempt the upgrade to 4.2. ( or have miss understood the above “Edit on the upgrade.php” or is this check is already built in to 4.1.1)

      • Ipstenu (Mika Epstein) 4:00 pm on April 13, 2015 Permalink | Log in to Reply

        The checks are built in.

        > Your site will only upgrade when the following conditions are met:…

        That’s what’s doing it :) If any condition fails, no DB changes. If you’re not even on the right version of MySQL, it won’t do it either.

    • snowboardmommy 12:30 am on April 24, 2015 Permalink | Log in to Reply

      FWIW, I upgraded a multisite installation (with debug mode ON) to 4.2 today and also found a similar DB error message in the error_log in wp-admin:

      WordPress database error Duplicate key name ‘meta_key’ for query ALTER TABLE wp_usermeta ADD KEY meta_key (meta_key(191)) made by wp_upgrade, make_db_current_silent, dbDelta
      WordPress database error Duplicate key name ‘domain’ for query ALTER TABLE wp_site ADD KEY domain (domain(140),path(51)) made by wp_upgrade, make_db_current_silent, dbDelta
      WordPress database error Duplicate key name ‘meta_key’ for query ALTER TABLE wp_sitemeta ADD KEY meta_key (meta_key(191)) made by wp_upgrade, make_db_current_silent, dbDelta
      WordPress database error Duplicate key name ‘domain_path’ for query ALTER TABLE wp_signups ADD KEY domain_path (domain(140),path(51)) made by wp_upgrade, make_db_current_silent, dbDelta

      I haven’t peeked at the database (phpMyAdmin) yet, but everything appears to be working alright.

    • jalev 8:01 am on April 24, 2015 Permalink | Log in to Reply

      Hi Gary, I did the upgrade to 4.2 and afterwards it asks me for a database upgrade as I obviously belong to one of the cases you described above. The problem is that the Database Update process timeouts.

      I have increased the timeout to 60 minutes but there is an alter query that after almost 15 minutes of running, it crashes my server as it takes all the ram and swap available. The query is the following: ALTER TABLE nB8KH50Wbb_options CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci

      and below it there is a huge que of locked queries as they try to write in the same table “Waiting for table metadata lock”.

      Any ideas?

      • Ipstenu (Mika Epstein) 12:06 pm on April 24, 2015 Permalink | Log in to Reply

        How big is your db? How big are your post and post meta tables?

        • FolioVision 8:31 am on May 7, 2015 Permalink | Log in to Reply

          Hello Mika,

          we had the same issue, we had to do this with a command line PHP, see our link in other comment here.

          Here is some info about the two sites where we had the issue:

          Site 1: wp_posts 256MB, wp_postmeta 57MB, wp_comments 292MB, wp_commentmeta 181.9MB, wp_options 19.8MB
          Site 2: wp_posts 84.9MB, wp_postmeta 5MB, wp_comments 134MB, wp_commentmeta 173.7MB, wp_options 9MB

          Perhaps you should improve the upgrade process to remember which queries finished before it times out. When it times out, it seems to be executing all the queries again, even if they finished before.


          • jstensved 4:26 pm on May 22, 2015 Permalink | Log in to Reply

            I can confirm that I’m having the exact same issue on another large site which I can’t update. I’m updating from command line with WP-CLI but still the command will eventually consume all memory.

    • l3hworks 4:00 pm on April 24, 2015 Permalink | Log in to Reply

      Hello, I updated to 4.2 in xampp and now I can’t go back to my life server (mysql 5.1), is there any way I could revert this?

    • Nihad 4:46 pm on April 24, 2015 Permalink | Log in to Reply

      I’ve upgraded to 4.2 today, on two different domains, one is MultiSite (Danish) other one (in English) is not.
      I’ve got no errors, so all is good. But my encoding is wrong. Content is not being displayed properly, even if it is created and saved as UTF8. On both sites.

      Since then, in troubleshooting, I’ve manually converted my db/data/tables from UTF8 to UTF8MB4, to see if this will resolve the issue. Nope.
      Iconv utility converting data to UTF8 just in case it was not. Did nothing either.
      My data looking at it directly in mysql looks good. Even Sequel Pro displays data correctly.

      It worked perfectly on pre 4.2… now it does not.
      Posts that are incorrectly displayed are non editable in WordPress either. It show, empty titles and contents. and when you save them, they are empty. So it’s like it can’t recognize encoding and then it just saves it as empty.

      But… If i copy data directly from database, paste is into a post, that is displayed as empty, in WordPress editor, and save it… It works correctly. So already existing data is somehow read incorrectly, but if the same data is copied into the editor and save to database it is good.

      Seems like there is an issue, in updating of database that went wrong somewhere.

      • sistemasBebetter 6:51 pm on April 29, 2015 Permalink | Log in to Reply

        Have you found any solution to this? I’m having the same problem but only with one of the sites I have (the others in the same computer and database upgrade correctly).
        Which WP version had your sites before upgrading?

      • watomsk 7:50 am on May 3, 2015 Permalink | Log in to Reply

        Did you manage to find a solution to this? I upgraded to 4.2 and have the same problem with my website.

    • neo7 12:58 pm on April 26, 2015 Permalink | Log in to Reply

      Got this error while updating “PHP message: WordPress database error Specified key was too long; max key length is 1000 bytes for query ALTER TABLE wp_commentmeta CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci made by wp_upgrade, upgrade_all, upgrade_420, maybe_convert_table_to_utf8mb4”

      Now all tables are utf8mb4_unicode_ci except wp_commentmeta which is utf8_general_ci.

      How to covert this table?

    • interglobalmedia 1:47 am on April 27, 2015 Permalink | Log in to Reply

      Hi Nihad. I had kind of a similar issue when I updated to 4.2. What is actually happening is that incorrect code that was previously undetected or slipped under the radar so to speak, has now been detected and then “erased”. I’ll give an example. I was working with a theme that someone else had begun developing, and there was incorrect code in it. I hadn’t gone over everything yet, and did not realize that the theme’s sidebar had not been registered or configured properly. So, when I updated to 4.2, I was told in the backend that my sidebar’s id had been incorrectly configured. Could I please manually correct it? I forget exactly what I did, but when I did it, the sidebar disappeared. What was good about this and I saw as a real plus, was that it told me what was wrong, and then I knew what I had to do to fix it. i fixed it. Luckily, it was on a local install. Of course. I was still developing it. Then I had another issue post 4.2 that had never come to the fore previously. I have a theme that uses a page template called “full width content”. However, I had purchased a premium plugin called “Ultimate Landing Pages” back in February. It wasn’t quite clear what sort of template the developer used for these landing pages, but everything seemed to work just great. Until 4.2. Today, I went into one of my site’s regular pages that I had configured to “full width content a couple of months ago, and I noticed that it no longer ran the full page width that it had before 4.2! The now invisible sidebar prevented the content from running the full width of the page. I thought it bizarre and went into the Editor to see what was going on. The Ultimate Landing Pages plugin’s template finally had shown up. It was configured to the “Front Page Template, which is one of the WordPress “Reserved Theme File Names”, has special meaning for WordPress, and if present along with a custom theme template that does the same thing, it will take precedence. So the fact that I had both the Front Page template and the Full Width template which both displayed the page in the same way, caused the Full Width template to no longer display “properly”. I had to go into the Editor and change the page template to “Front Page”. This partially has to do with cache issues, but also 4.2 lets you know either overtly (or covertly) about incorrect code/configuration issues. I am going to try and find out from the WordPress core developers what changes they made that made this possible. I see it as a very positive thing. Hope this helps.

    • DrLightman 10:40 am on April 27, 2015 Permalink | Log in to Reply

      Could this index upgrade fail or time out on databases with large tables, let say 250’000+ posts thus lot more postmetas?

      If we do this upgrade manually let say via command line on dedicated servers, will WP 4.2 skip the upgrade during installation?

    • jtleathers 9:56 pm on April 27, 2015 Permalink | Log in to Reply

      Developing a site locally on WAMP with SQL 5.6.17. When I attempt to import the database to a server using an older version of SQL I get the following error:

      – Unknown collation: ‘utf8mb4_unicode_ci’

      How do I properly rollback this “upgrade” from WordPress 4.2?

      • Ipstenu (Mika Epstein) 11:30 pm on April 27, 2015 Permalink | Log in to Reply

        Import how? Phpmyadmin?

        • mctenold 11:55 pm on April 27, 2015 Permalink | Log in to Reply

          Same issue here – importing any way, CLI, phpMyAdmin, or personally I’m using MySQL Workbench for Mac and getting the same errors. For now I’m going to have to not use 4.2 until I can find a workflow to go from Local -> Dev Server, where local MySQL is compatible with 4.2 but dev server is not.

          • Max Sixblade 5:16 am on May 4, 2015 Permalink | Log in to Reply

            True! Using XAMPP for Windows, this is definitely not a phpmyadmin issue. Older mysql version on remote server leads to errors while importing db. Compatibility modes while exporting db didn’t help in my case.

        • jtleathers 12:45 am on April 28, 2015 Permalink | Log in to Reply

          Yeah, import through PHPMyAdmin.

        • MBWD 3:23 pm on May 16, 2015 Permalink | Log in to Reply

          I am having the same problem with my shared hosting server, which is using an older version of PHP. The upgrade to 4.2 apparently went fine for sites already hosted, but importing a db with utf8mb4 doesn’t work. I tried downgrading my version of WP to 4.0 and upgrading the database (which I thought would revert the charset) but it didn’t work. Manually replacing didn’t work either.

    • mctenold 12:54 am on April 28, 2015 Permalink | Log in to Reply

      I’m in the same boat as @jtleathers, my typical workflow is to install WordPress locally for a new site, build the site locally, and then move everything to a dev or production server once I’m at a good place.

      The issue comes in when my client’s dev or production server has a MySQL version 5.5.3. I get the same error upon attempting to import the database: Unknown collation: ‘utf8mb4_unicode_ci’.

      This seems like a misstep of an upgrade, I think my workflow is pretty common practice and will affect a lot of people. I don’t think it’s common to install/develop the site on the production server.

      Any remedy for my dilemma?

      Seems like the only possible fix right now is to downgrade my local MySQL version, or not use WordPress 4.2.


    • dustinbolton 10:20 pm on May 6, 2015 Permalink | Log in to Reply

      BackupBuddy developer here: It is absolutely INCORRECT to say that migrating sites to new server is uncommon. BackupBuddy helps users move several hundred thousand sites per year to new servers.

      We are already seeing a flood of users encountering this problem attempting to move their site to new servers with older MySQL versions. They of course are puzzled why this is a problem since WordPress states it supports this older MySQL version.

      Many users these days practice things such as local development, staging, etc which are good modern practices. WordPress has effectively put a huge thorn in the sides of developers and in my opinion has made an extremely big mistake.

    • lucaboccianti 6:38 pm on May 11, 2015 Permalink | Log in to Reply

      I may have found a possible walkaround.

      I’m in the same situation and problem as above: develop locally with xampp then upload on the production server (mostly shared hosting).

      with phpmyadmin for each table edit each field codified with utf8mb4-something to be codified as utf8-something (select the table, then Structure, etc.). then edit the table properties (Operations) and do the same.

      finally export the dump and restore it on the production server.

      it’s long and painful but maybe better than restart a project from scratch.

    • lucaboccianti 6:49 pm on May 11, 2015 Permalink | Log in to Reply

      problems: you’ll end with all the extended characters looking funny and apparently the text edit area appear blank even if the text is visible on the frontend.

    • Max Sixblade 11:36 pm on May 11, 2015 Permalink | Log in to Reply

      So.. where shall we vote to get that “install in default UTF-8” checkbox on WP installation?)

    • Ben Lobaugh (blobaugh) 1:44 pm on May 12, 2015 Permalink | Log in to Reply

      I have had several migrations happen this week, and even between the dev, staging, and production servers. Ran into lots of issues and what I wound up doing is running a script to convert the database back to utf8. This worked great to get the data imported and the next time wp-admin was loaded the database was successfully updated with the security patch.

      If interested here is the script I used http://ben.lobaugh.net/blog/201740/script-to-convert-mysql-collation-from-utf8mb4-to-utf8

      • dblomster 3:23 pm on May 12, 2015 Permalink | Log in to Reply

        Nice! What about table indexes and such? Is there anything else that need to be changed when going from utf8mb4 to utf8 again?

        • Ben Lobaugh (blobaugh) 4:54 pm on May 12, 2015 Permalink | Log in to Reply

          All I did was run that and it magic worked. Before running that script the db was complaining when I tried to import via sql files.

        • MBWD 4:00 pm on May 16, 2015 Permalink | Log in to Reply

          Hi Ben, I’m also on a Mac and have saved this script you so thoughtfully provided as a .sh file to run through Terminal, but when I run it I am getting an error saying “mysql: command not found”. Any idea what could be wrong?

      • dotwongdotcom 4:40 pm on May 12, 2015 Permalink | Log in to Reply

        Ben! Thanks for this! You may have just saved me a huge headache. I stumbled upon your post through a rabbit hole of clicking while trying to figure out a workaround for errors that I was encountering while using the migration tool BackupBuddy. My local dev is running WP 4.2, but the migration won’t allow utf8mb4 on the live site because their version of mysql. I’m a relative novice when it comes to these things… how would I implement your script to convert my local build’s tables back to utf8 so that I can migrate to my live server?

        • Ben Lobaugh (blobaugh) 9:09 pm on May 12, 2015 Permalink | Log in to Reply

          Copy it into a bash script on your local dev. Edit the variables for db access and run it from the terminal. This assumes you are on a unix based system such as Linux or OS X. I do not have Windows to know how to build a script for that.

    • zanozik 1:38 am on June 23, 2015 Permalink | Log in to Reply

      Server version: 5.5.19-cll
      MySQL client version: 5.5.33
      Upgraded to 4.2.2 and my wp “crashed”. Every option that was in unicode was either unavailable or missing non-latin chars. Pages and posts content was unviewable. Titles were gibberish.
      Since I use a web hosting (no root, or cpanel access) with remote phpmyadmin I logged in there to see that all of the tables collation have been changed to utf8mb4-general. Database collation was still utf8-general, strange, I thought.
      The hosting I use, let a user create/change a database in custom interface only. Rather popular solution. So I thought maybe the upgrade was only partly successful and the database was not charset was not actually upgraded, just collation changed. So I changed charset in wp-config, changed collations for each tables, and voalia – everything working again…
      Could that buggy force upgrade happen to “omg-so-professional” WordPress? I just dont know what to do, laugh or cry right now…

    • physalis 11:20 pm on July 2, 2015 Permalink | Log in to Reply

      @zanozik I had the exact same problem, only when I changed from a dev to the final server – every little special character was nuked, plus options and posts in the backend where cracked. I tried setting my dev server database, but to no avail (so I already thought it must be the files then).
      Your description helped me save a release date for that project tomorrow morning and thus quite some people around it ;). So thanks a bunch!

    • treavioli 8:34 am on July 10, 2015 Permalink | Log in to Reply

      I encountered this issue when I tried to export my database tables via phpmyadmin. I would get several errors and warnings about collation. I switched to Sequel Pro and exported/imported that way, but I’m still not without issues. My post editor won’t recognize posts/pages/etc I created before site migration. Nothing appears in the edit panel for either Visual or Text tab..

  • Gary Pendergast 1:26 pm on February 25, 2015 Permalink
    Tags: , ,   

    Emoji Chat Agenda, February 25, 2015 

    Here’s the agenda for Wednesday’s Emoji Chat in the #core channel on Slack.

    Time/Date: Immediately after the Dev Chat.

    1. Emoji Helper – On platforms that don’t have an emoji keyboard, should we provide one?
    2. Performance – When we’re falling back to Twemoji on large pages, we have to consider how it will perform. Are there faster ways of replacing the emoji characters with images?
    3. Open Floor – There’s still time to rant about the evils of emoji! Let us know how you really feel.
  • Gary Pendergast 12:26 am on February 13, 2015 Permalink
    Tags: ,   

    Emoji Chat Meeting Notes, February 12, 2015 

    The full meeting archive is available here.

    1. Why we’re doing this

    So, here’s a bit of back story.

    As of r31349, WordPress partially supports emoji. ~60% of WordPress sites are running MySQL 5.5 or later (so can be upgraded to store emoji), and ~40% of browsers natively support emoji. Emoji are a wildly popular method of communication, so we can expect them to be heavily used as soon an they’re available. The problem is, 60%/40% means a really bad experience for a huge number of our users, who’ll try to use emoji, and fail.

    This is where the emoji feature plugin comes in to play. In order to help the 40% of WordPress sites that can’t be upgraded to store emoji natively, the wp_encode_emoji() function will turn them into HTML entities. Due to the unimaginable joy that character sets brings me, this will only be applied to sites using the utf8 character set, which accounts for the vast majority of WordPress sites – utf8 has been the default character set since r4860.

    To help the 60% of browsers that don’t display emoji natively, we’re using the Twemoji image set as a fallback. This lets us show emoji everywhere, without causing extra load where emoji are already supported, mobile browsers being the important example here.

    Now, there have been some concerns brought up previously that I’d like to address.

    “Is this really appropriate for core?”

    Yes. (Obviously that’s my answer, or we wouldn’t be here.) WordPress is is the business of making communication simple and accessible for all. Tech users everywhere have clearly chosen emoji as a means of communication, so it’s up to us to make sure they can do that within WordPress as easily as possible, or risk being left behind.

    “Should we be concerned about changing the images in the future? Wouldn’t we be altering users’ content?”

    No. By using Twemoji only when we can’t provide native support for emoji, it’s a pretty clear message that while the general appearance of emoji stays the same, the actual sprite used can differ significantly between platforms. (For example, every emoji set except Android uses a left hand for :thumbsup:.) As more browsers add native support for emoji, Twemoji usage will drop, reducing even further any impact we can have on users.

    And so, that brings us to today.

    2. The current state of the plugin

    The plugin is very close to done. The editor plugin needs some attention, which @azaozz will be providing soon. There are a few bugs to discuss, which are mostly around fallback behaviour in browsers that don’t support emoji natively. Apart from that, the basic functionality is pretty much how I would expect it to appear in core. It’s had a brief review from the accessibility team, with only some minor alterations needed. The Twemoji images won’t be included in wordpress.zip, as it’s a total of 3.4MB of images. They’re currently hosted on WP.com’s CDN, but we’re investigating other options for where to host them, probably the W.org CDN. Given that the wp-admin Dashboard also loads things from Google, I have no problem with hosting them on an external CDN. There will naturally be a filter on the URL, to allow local hosting for sites that don’t want to use the CDN.

    One of the major concerns at the moment is that we’re going to be splitting data formats, depending on if the site uses the utf8mb4 or the utf8 character set. utf8mb4 stores emoji natively, while utf8 requires us to HTML encode the emoji characters. In the futures, we’ll look at upgrading sites to utf8mb4 if they’ve upgraded their MySQL since WordPress 4.2, but that leaves the potential for mixed encoding – old posts having HTML encoding, new posts having the native characters. A post will be automatically updated to native upon saving, but do we need to consider upgrade routines, to go through all old posts and convert them?

    Export/import also needs thorough testing, particularly when importing and exporting between sites having different character sets.

    3. Unicode 8.0: the future of emoji

    To talk about the future of emoji, you need to know a little bit of history. At the basic level, emoji are all single characters defined within the Unicode standard. However, they also support modifiers. Modifiers are a second character following the first, which usually causes the two characters to be merged into a single character when rendered. A good example of this is flag emoji.

    The character G is U+1F1EC. The character B is U+1F1E7 (these characters are different to their ASCII equivalent). When used individually, they’ll display as that letter. When combined next to each other, they’ll display as the British flag.

    So, Unicode 8.0 will two interesting things: a set of 37 new emoji, and skin tone modifiers. When a skin tone modifier character is placed after any face or person emoji, the emoji will show with that skin tone. Unicode 8.0 is due to be finalised in August 2015, so we and (Twemoji) will be looking at adding support for these then.

    From a technical perspective, it just means we need to be aware that emoji are not always one character, and the methods for detecting multi-character emoji are about to get more complex.

    We’ll also be able to detect if a browser is able to render the new emoji and skin tones, and fall back to Twemoji if they can’t. I don’t have a timeline for when browsers will support the new emoji, so I think it’d be good for us to get ahead of the curve then.

    utf8mb4 stores anything in the Unicode address space, including unallocated characters, so I don’t expect any problems with storage of new emoji.

    • Ryan Hellyer 8:45 am on February 18, 2015 Permalink | Log in to Reply

      I’m not seeing any reason to host them externally. Why not just bundle an extra 3.4 MB into the zip?

      Even better, perhaps this should just stay as a plugin? Maybe I’m wrong, but this seems a very niche thing to me.

      • John James Jacoby 1:17 pm on February 18, 2015 Permalink | Log in to Reply

        Particularly in the US, I can anecdotally tell you that Emoji has had slow adoption for several years, but it’s becoming less-so with the native inclusion of it in iOS. See my comment below about smilies, which also very easily could have been a plugin (and may have originally been at one point?) but they’re so simple and commonplace now, it’s weird not seeing them rendered as images. :)

      • Gary Pendergast 8:26 pm on February 18, 2015 Permalink | Log in to Reply

        Why not just bundle an extra 3.4 MB into the zip?

        They’re compressed PNGs, so they won’t compress much further. They’d be doubling the size of the zip.

        Maybe I’m wrong, but this seems a very niche thing to me.

        Not so much. Emoji are a very popular method of communication by large groups of the the population. If anything, I’d say it’s an oversight that it’s taken us so long to get to supporting emoji.

    • John James Jacoby 1:15 pm on February 18, 2015 Permalink | Log in to Reply

      Something I don’t see mentioned is the current iteration of smilies.

      • Does Emoji support secretly allow us to nix old-school smilies and cut out an option?
      • Can/should Emoji support be tacked on-top-of smilies and work similarly to Slack and use a bunch of string-replacements?

      Understanding the approach and code behind each is different, they are identical to a typical end-user because – rather conveniently for us – smilies in core have no UI (though there are plugins that implement one.)

      I’m imagining either a new UI option for Emoji similar to smilies in writing settings, or having it use that same one, which led me to wondering how dissimilar the underlying functionality needs to be, or if this is an opportunity to update an old API.

      • Gary Pendergast 8:33 pm on February 18, 2015 Permalink | Log in to Reply

        Does Emoji support secretly allow us to nix old-school smilies and cut out an option?

        If I told you, it wouldn’t be a secret anymore, would it? 😉

        Can/should Emoji support be tacked on-top-of smilies and work similarly to Slack and use a bunch of string-replacements?

        Nope. That’s a pretty niche usage, with native OS and browser support for emoji increasing, I’d prefer to keep that kind of hack out of core. It’d also be a super expensive search/replace to have happen at render time.

        Currently, there’s no plan to change/remove the existing smiley option – emoji will just work if you happen to use them, smilies will continue to be optional (but will be replaced by nicer images if you do use them).

        I would like to deprecate the smiley option at some point, but I haven’t given it enough thought to say when that could/should happen.

    • Paal Joachim Romdahl 11:54 pm on February 18, 2015 Permalink | Log in to Reply

      It seems adding emoji to core will cause some perhaps a lot of frustrations for people:

      As the comment seems to be they can add that feature to core but what about…..-insert feature-

      • Paal Joachim Romdahl 1:24 am on February 20, 2015 Permalink | Log in to Reply

        Another thought….. Call it a Easter Egg. Like in games that have hidden features. Press a combination of keys or go and do something special in a game and it unlocks something cool.
        I think each release of WordPress could include a Easter Egg (“just” for fun and unique use). Emoji is the first Easter Egg and fits perfectly as Easter is around the time of the 4.2 release.

        So when people ask why it was included in WordPress 4.2 one can say it is an easter egg. Meaning it was included “just” for fun (of course it has it purpose for being included).

  • Gary Pendergast 11:25 pm on February 11, 2015 Permalink
    Tags: ,   

    Emoji Chat Agenda, February 12, 2015 

    Here’s the agenda for Thursday’s Emoji Chat in the #core channel on Slack.

    Time/Date: February 12 2015 23:00 UTC

    1. Why we’re doing this – @pento
    2. The current state of the plugin – @pento
    3. Unicode 8.0, and future plans – @pento
    4. Open Floor/ranting about the evils of emoji – everyone

    See you tomorrow!

  • Gary Pendergast 3:07 am on February 9, 2015 Permalink
    Tags: ,   

    Emoji Feature Plugin for 4.2 

    It’s time for a weekend fun feature! Now that #21212 is complete, WordPress kind of supports Emoji (for the 60% of WordPress sites using MySQL 5.5+, and the 30-40% (by usage) of browsers that natively display Emoji – including when Chrome for OS X adds support in the next month or so).

    In order to complete this support, I’ve created a feature plugin called x1f4a9, which makes use of Twitter’s Open Source twemoji icon set, the same as WordPress.com recently added.

    I’ve added a few tickets to the Github project, feel free to add any others you think of, and pull requests are always welcome! If you’d like to test the plugin, daily builds are available from the plugin repo.

    (And if you’re using MySQL older than 5.5, please pay special attention to this ticket.)

    • Aaron Brazell 3:20 am on February 9, 2015 Permalink | Log in to Reply

      We can do this but we can’t do Postgres support? We can do this but dynamic image sizes is plugin territory?

      • Gary Pendergast 3:59 am on February 9, 2015 Permalink | Log in to Reply

        That’s correct! :-)

        Postgres has very low usage – the plugin has < 1000 installs. The massive amount of rearchitecture it would take to support it in core is totally not a viable option.

        Dynamic image sizing is a very CPU intensive task, which would make your average shared host fall over. We don't really want to make every host explode, so it stays in plugin territory – easily usable by folks who have the server power, or as a service like Jetpack's Photon.

        And because I've been working on architecture things for the last 6 months or so, I get to choose to work on some fun things every now and then. 😉

        • Aaron Brazell 2:38 pm on February 9, 2015 Permalink | Log in to Reply

          Certainly no one wants to tell you what you can or should work on. I question its’ existence as a featured plugin though. I mean, we could make my Superhero Avatars a featured plugin based on your argument. 😉

          Really, not trying to be argumentative, but this isn’t the progress that I, for one, think WordPress core should be pursuing (and I do understand that featured plugins are not core, but they are meant to be potentially core material down the road).


          • George Stephanis 2:55 pm on February 9, 2015 Permalink | Log in to Reply

            It’s easy to fall into groupthink, but the fact of the matter is that a lot of users want to use Emoji, and Core’s inability to support that is a stumbling block for many users.

            Good software should work out of the box.

      • Helen Hou-Sandi 4:32 pm on February 9, 2015 Permalink | Log in to Reply

        We’re talking about support for characters that are available in a standard and standardized, easily and frequently accessed keyboard palette on a non-insignificant percentage of devices, both desktop and mobile. Also, while yes WordPress is a project and there are overarching directions and initiatives, most things are still worked on by individual contributors. Belief that X should be worked on instead of Y is largely a fallacy, because most plugins and patches are whatever a given contributor is both inspired to and has the expertise to work on. In this case, this is a natural extension of the exhaustive work Gary has done on the architectural support front for the last few months.

      • petermolnar 9:14 am on February 10, 2015 Permalink | Log in to Reply

        You’ve spoken from the heart of many.

    • Chris Van Patten 4:58 pm on February 9, 2015 Permalink | Log in to Reply

      I took a quick look at the feature plugin and like most of it… the wp_encode_emoji function is clever and it all looks nice, but I’m really don’t like that it uses Twemoji. I understand why the Twemoji approach is gaining popularity, but it seems like a waste to hack in the emoji support when browsers are on track to get it implemented themselves. On browsers that do support emoji, this approach causes unnecessary file requests and deviates from the system’s standard emoji set. Then what happens when browsers finally do support it en masse? You can’t really remove the support without opening up to user complaints that the emoji images have changed and inevitably screwed up some aspect of their site’s design.

      I think all the technical fixes in the plugin make sense (encoding the emoji and whatnot) but I really think the Twemoji stuff should either be an opt-in setting or a separate non-core plugin.

      One dude’s opinion, anyway :)

      • Gary Pendergast 12:58 am on February 10, 2015 Permalink | Log in to Reply

        I think there are good arguments either way. As you mentioned, it’s a bit weird to override native sprites, and it does have the potential to cause backwards compatibility issues. There’s a little bit of extra data to be downloaded, sure, but I think it’s worth the tradeoff. In the same way that most sites load jQuery, it’s a bit more data, but provides a much better experience for everyone involved.

        I think it’s important to look towards provide a consistent experience across all WordPress sites. WordPress being as big as it is, I think we qualify as our own ecosystem, we can look to provide a smoother experience, regardless of platform.

        I also think we’ll run into problems with inconsistent emoji support in different platforms that current support emoji. Unicode 8 (due to be finalised later this year) defines a bunch of new emoji, which are currently not supported on any platform. It also defines skin tone modifiers for existing emoji, which are trickier to implement, so will potentially delay the rollout on various platforms. Depending on how long it takes to draw the new images, we could be rolling out Unicode 8 support before anyone else.

        Finally, we have an existing relationship with the Twemoji project, through their partnership with WordPress.com. They’ve been great people to work with on that project, I’m optimistic that WordPress core can have a great relationship with them, too. :-)

        • Chris Van Patten 6:19 pm on February 10, 2015 Permalink | Log in to Reply

          I don’t want to come across as being altogether anti-Twemoji as I think it’s a really interesting approach and clearly it’s well done for what it is, but I think it’s worth bearing in mind that it’s a bit of a different case for WordPress.com and Twitter vs self-hosted WordPress.

          It would be easy for Twitter (for instance) to decide that browser support is Good Enough™ and subsequently disable Twemoji on their platform, because they fully control the user experience and can verify it works. Same for WordPress.com (to a lesser extent)—the themes are tightly controlled and it’d be trivial to write a regression test before disabling Twemoji to verify it won’t break things. For self-hosted WordPress, there are so many other moving pieces, and really once you turn this on you kinda can’t turn it off. Even if all major browsers fully support emoji in all its forms within a year or so (optimistic, sure; but I do think we’ll see really high adoption rates of most emoji features within a year), you’re kinda stuck, because who knows what might break for the users if you turn it off.

          I guess ultimately I’m just really uncomfortable because it just feels like a hack. Emoji are supposed to be *fonts*, not images. That’s by design. Really, the whole image replacement technique reminds me a lot of the old SIFr type replacement system, that replaced text with a Flash embed so you could use custom fonts. Sure it was more consistent, but it was also slow and annoying. But it was also okay because it was something only devs implemented, and it was easy for the dev to switch to web fonts once they received wider browser support.

          Here, it’s making that decision on behalf of users who might not fully understand the implications.

          Just to close things off I really don’t want to come across as negative and anti-emoji and whatever. I just think the Twemoji approach might not be the best—or at least not as a platform-wide forced-on solution.

          (I’d also say that in regards to the additional HTTP requests, many themes and plugins are already *really bad* about this; we’ve all seen the random WP site with 30 plugins loading dozens of JS dependencies and the poorly developed themes with never-ending tags. With that in mind, I really think adding additional platform-wide HTTP requests—for emoji or anything else—should be a really deeply and seriously considered issue. We’re pretty lucky in the developed world to have relatively fast Internet where additional requests aren’t that noticeable, but for users in the developing world, or users in rural areas without access to reliable broadband, those requests add up fast. This is goes for mobile users across the world as well. It already sucks waiting for web fonts to load on a 3G network… why add emoji to that too when most modern mobile browsers support emoji natively?)

          • Gary Pendergast 10:59 pm on February 10, 2015 Permalink | Log in to Reply

            That’s a good argument, I absolutely agree that slow connections and mobile rendering should be taken into account.

            There seems to be a way to detect if emoji is supported in a browser or not (rather than maintaining a list of browsers that support emoji). I’ve added a ticket to track it here:


      • Gary Pendergast 12:59 am on February 10, 2015 Permalink | Log in to Reply

        Oh, and thank you for noticing wp_encode_emoji(). I’m just a little bit proud of that bit of code. 😀

    • Lara Littlefield 9:25 pm on February 9, 2015 Permalink | Log in to Reply


    • Michael Arestad 6:16 pm on February 10, 2015 Permalink | Log in to Reply

      As much as I love emoji, I don’t know that they should be in core. The reason is that once these make it into core, they will stay in core forever because they would be considered user content. This means, that should we update this set ever, there would need to be controls for versions of sets (as well as bundling these images forever) or we’d be messing with users’ content.

      • Gary Pendergast 11:01 pm on February 10, 2015 Permalink | Log in to Reply

        Assuming we only load Twemoji on browsers that don’t natively support emoji, I think that would do a good job of setting expectations that emoji look different on different browsers. That makes their appearance trimming, rather than content.

      • BizzThemes 10:31 am on February 11, 2015 Permalink | Log in to Reply

        Good point. Same should apply to Press This. WordPress is not a social network and doesn’t need these social networking tools.

        • John Blackbourn 9:45 pm on February 11, 2015 Permalink | Log in to Reply

          What’s this got to do with social networking?

          • BizzThemes 8:28 am on February 13, 2015 Permalink | Log in to Reply

            Press This is a sharing feature, no? FB has it, TW has it, but not sure why WP.org needs this in core. Sharing tools work great on social networks, where content consumption is organized around sharing stuff from the web. But WP.org is not a social network. We should also support original content. Imagine, who would read Medium if there were re-posted articles on it.

    • Piet Bos 7:44 am on March 25, 2015 Permalink | Log in to Reply

      I won’t go into whether or not I like this. My problem with the emoji feature is that it adds javascript to the footer of my site. Apart from the fact that I don’t want to add unnecessary calls for things I don’t use, one of the scripts is calling the URL http://s0.wp.com/. This URL is blocked in China where I currently reside, which means that my own site now takes forever to load, because it first has to wait until the call to http://s0.wp.com/ times out.

      This cannot be the meaning of progress…

      Is there a way to turn this mess off for people that don’t want it, or do those people have to jump through all kinds of hoops to disable the lot?

    • Piet Bos 8:08 am on March 25, 2015 Permalink | Log in to Reply

      Fortunately I found a way to disable the lot. Can’t add it to my earlier comment as that is still under moderation, but for anyone not interested in this “feature” and/or lives in a location where s0.wp.com is blocked, have a look at https://wordpress.org/plugins/disable-emojis/

    • Rene Hermenau 4:11 pm on April 13, 2015 Permalink | Log in to Reply

      Or better use the one by otto: https://wordpress.org/plugins/classic-smilies/

      I stumbled over this thread when i was looking for a way to disable this new feature as i dont need any emoj handling for my pages. Could you please give user the possibility to disable this feature completely?

      Imho it should be part of the WordPress codex:
      New core functions which are not essential must have a disable function.


      Its not much kind amusement to have the need for another plugin only to disable a non-essential “fun-feature”;-)

      • xwolf 12:17 pm on April 24, 2015 Permalink | Log in to Reply

        Yes, please!
        “New core functions which are not essential must have a disable function.”

    • LuxDelux 1:10 pm on April 24, 2015 Permalink | Log in to Reply

      Simply adding an option to disable Emoji functionality would solve most negativity.

      I mean there’s an option to enable/disable emoticon conversion in posts… why not a checkbox under it do enable/disable emoji.

      Keep it checked by default if most users deem it useful but enable others to disable it.

      Removing 4 actions and 3 filters via code or plugin is just nasty.

    • webliberty 9:11 pm on April 24, 2015 Permalink | Log in to Reply

      For the first time in several years I had to roll back a new version of WordPress to the old one. The Reason Is Emoji. What was the need to implement this for all users by default? That’s a pretty specific function! I want the old smilies!

      Give the opportunity to choose to use emoticons or Emoji. After updating all my emoticons partially replaced by Emoji, partially instead of black squares that looks terrible!

      Previously, I have added all the smilies under the comment form that commentators would put them in the form of a comment, now it will not start! Still the file names in the path to the images are broken, replacing the file name in Emoji and a link causes a 404 error. Do something, it looks bad.

      I suggest to remove Emoji from the core of WordPress and add them as a plugin, for example in a Jetpack.

    • Jeff Sebring 5:10 pm on May 10, 2015 Permalink | Log in to Reply

      Real sneaky geniuses at work here. I just watched Mr. Nacin’s talk about this at Loop Conf. Thanks for all you have done.

  • Gary Pendergast 6:20 am on April 7, 2014 Permalink
    Tags: , , , ,   

    MySQL in WordPress 3.9 

    In WordPress 3.9, we added an extra layer to WPDB, causing it to switch to using the mysqli PHP library, when using PHP 5.5 or higher.

    For plugin developers, this means that you absolutely shouldn’t be using PHP’s mysql_*() functions any more – you can use the equivalent WPDB functions instead.


    There are a few different options for replacing the query functions, depending on what you want to do:

    As a drop in replacement to run a query that you don’t expect a return value from (i.e., an INSERT, UPDATE or DELETE query), use $wpdb->query(). This will always return the number of rows effected by the query.

    Alternatively, $wpdb->insert(), $wpdb->update(), $wpdb->delete() and $wpdb->replace() are all helper functions that will automatically escape your data, then generate and run the queries for you. Ideally, you should never need to write an SQL statement!


    If you have a SELECT query, for which you’d normally do a mysql_query() followed by a mysql_fetch_*(), WPDB lets you combine this into one function call.

    To get all of the results from a query that returns more than one row, use $wpdb->get_results() to return an array of objects containing your data.

    There are also some shortcut functions for common usage:

    If you only need a single row from your query, $wpdb->get_row() will return just the data object from that row.

    If you only need a single column from a single row, $wpdb->get_var() will return only that field.

    And if you need a single column, $wpdb->get_col() will return an array of all the data from that column.


    For a drop in replacement, you can use esc_sql(). That said, we strongly recommend switching to $wpdb->prepare(), instead. We have a pretty thorough tutorial available for $wpdb->prepare().


    If you need to get the Insert ID from the last query, $wpdb->insert_id is where you need to look.

    Updating your plugin to use WPDB will also future proof it for if we make changes to how WordPress connects to the database – we’ll always maintain backwards compatibility with the current WPDB interface.

    For more reading, check the WPDB Codex page, and #21663.

    If you’re using MySQL in a way that I haven’t covered here, please post it in the comments, we’d be happy to help you out!

    • Samuel Wood (Otto) 6:23 am on April 7, 2014 Permalink | Log in to Reply

      Note: $wpdb->escape() is deprecated. Please use esc_sql() instead. Or $wpdb->prepare(), of course.

      Also note that $wpdb->escape() is not a proper replacement for mysql_real_escape_string() to begin with, as it only does weak escaping.

    • Doug Wollison 12:09 pm on April 7, 2014 Permalink | Log in to Reply

      I though I heard it was going to switch to PDO, not MySQLi?

      • Gary Pendergast 12:30 pm on April 7, 2014 Permalink | Log in to Reply

        That’s still on the cards for a future release, but it will be a significantly bigger project. The primary goal with this change was to stop using ext/mysql on PHP 5.5+, where it’s deprecated.

    • Brian Layman 4:10 pm on April 7, 2014 Permalink | Log in to Reply

      Wow! Huge change, and with the MySQL extension being deprecated in PHP 5.5 it’s a good one. I am trying to remember if 5.5 REQUIRES MySQLi to be built in. Does anyone know? Should you also throw in a check for the existence of mysqli_connect?

      Also, are there plans to start preparing queries? (Not WP prepare but the database meaning of prepare.)

      • Gary Pendergast 3:06 am on April 10, 2014 Permalink | Log in to Reply

        We do check that mysqli_connect() exists before trying to use it. (See wpdb::__construct().)

        There are no plans for adding statement prepare in the near future, though it is something I would like to get to!

    • webaware 10:46 pm on April 7, 2014 Permalink | Log in to Reply

      Plugin writers sometimes call mysql_get_server_info() to get the raw version information that $wpdb->db_version() strips out. Since the database handle isn’t public (nor is the mysqli indicator), there isn’t a way to do this in 3.9-beta3. I’ve opened a trac in hopes we can get a new method added to class wpdb so that plugin writers can still access this raw version information:


      • Gary Pendergast 5:28 am on April 8, 2014 Permalink | Log in to Reply

        To summarise the ticket, for anyone using mysql_get_server_info() – the best option is to have a switch in your code for mysqli, we’ll look at expanding $wpdb->db_version() at a later date.

        if ( empty( $wpdb->use_mysqli ) ) {
        	$ver = mysql_get_server_info();
        } else {
        	$ver = mysqli_get_server_info( $wpdb->dbh );
    • Claudio 8:51 pm on April 13, 2014 Permalink | Log in to Reply

      I use mysql_connect to test DB connectivity. How should I replace it for WP3.9/PHP5.5 compatibility?

      • Gary Pendergast 12:13 am on April 14, 2014 Permalink | Log in to Reply

        To test the connection, you can use $wpdb->check_connection(), which will check that the connection is up, and try to reconnect if it isn’t.

        This is particularly useful for long running cron jobs, where the MySQL connection might drop out due to inactivity, but there’s no actual problem with the server.

    • Ross Seddon 6:08 am on May 5, 2014 Permalink | Log in to Reply

      My web site which was designed using a standard off shelf theme I’m advised by our web site managers they site is providing the response “Access denied for user ‘www-data’@’localhost’ (using password: NO).” They advise
      Quote: “the one statement in there is exactly the issue: For plugin developers, this means that you absolutely shouldn’t be using PHP’s mysql_*() functions any more – you can use the equivalent WPDB functions instead.”

      The developer how wrote the plugin that is used site wide on your site for the skin, uses this no longer available method. All there work needs to be updated to use the correct method of connecting to the database. We know what is required, just that the plugin / skin is none of our work, and it is time consuming to fix.”

      We been down now for 3 weeks in terms of accessing site. Is what your referring to this thread relative to my site problem site http://www.totallyoutdoors.com.au

      how time consuming is this problem?


    • lwall 1:07 pm on May 15, 2014 Permalink | Log in to Reply

      I am having troubles with admin permissions at different points.

      When clicking on “Posts”, I get error “Invalid post type”, or when trying to create a new post, there is no save box/button.

      It also happens when trying to change options in one of my themes in other places with plugins. I get “You do not have sufficient permissions to access this page.”

      For the most part, the back-end is functional, the front-end is fully functional.

      I am using appengine 1.9.3 and wordpress 3.9, python 2.7.6.

      I have uninstalled 1.9.3 and updated to 1.9.4, I have also accepted WordPress’ request to install 3.9.1, the problem persists.

      I have installed the 1.9.3, 3.9, 2.7.6 configuration on a different machine where appengine was never installed before and the same problem occurs.

      I am discarding plugins because there were no plugins in the separate machine.

      I had appengine 1.9.0 and 1.9.3 working with WordPress 3.8.1. The problem started a few days ago after a number of upgrades (from 1.9.3 to 1.9.4, and into wordpress 3.9.1).

      Could this extra layer to WPDB be the source of this?

    • Michael Simpson 1:49 pm on July 11, 2014 Permalink | Log in to Reply

      For a plugin, I would like to be able to do unbuffered queries (MYSQLI_USE_RESULT). This is to handle cases when there are a lot of rows being returned and I want to keep the memory footprint down. wpdb doesn’t support this well and it would be nice if it would.

      For now, I have to create a new wpdb object and directly call mysqli_query($wpdb->dbh, $sql, MYSQLI_USE_RESULT); It is not using MySQLi, then I need to call mysql_unbuffered_query($sql, $wpdb->dbh);

      To know which one to call, I would like to consult $wpdb->use_mysqli but I cannot because it is private and there is no accessor method. So I have to copy the code from wp-db.php to determine it.

      For a start, it would be nice to be able to access $wpdb->use_mysqli. Even better would be an API on $wpdb to do unbuffered queries.


      • Gary Pendergast 2:02 pm on July 11, 2014 Permalink | Log in to Reply

        You can access $wpdb->use_mysqli directly, because of the wpdb::__get() magic getter.

        There’s unlikely to ever be an API in WPDB for doing unbuffered queries. There are too many gotchas that will just cause maintenance problems.

compose new post
next post/next comment
previous post/previous comment
show/hide comments
go to top
go to login
show/hide help
shift + esc
Skip to toolbar