Taxonomy term splitting in 4.2: a developer guide

In WordPress 4.2, shared taxonomy terms – those items in the wp_terms table that are shared between multiple taxonomies – will be split into separate terms when one of the shared terms is updated. This change ([31418]) fixes one of WordPress’s most irksome bugs, and is a critical step in our ongoing taxonomy roadmap.

Technical details and backward compatibility

Each row in the wp_term_taxonomy table represents the dyad of a term slug and a taxonomy – say, the tag slug ‘mint’ and the taxonomy ‘herb’. Historically, terms in different taxonomies that shared the same slug would share a single row in the wp_terms table; thus, the ‘mint’ row in wp_terms could correspond to a term-taxonomy pair in the ‘government_building’ taxonomy as well as ‘herb’. Starting with 4.2, when you update a shared taxonomy term – wp_update_term( $mint_id, 'government_building' ) – WordPress will detect whether the term is shared between multiple taxonomies, and if so, will create a new row in wp_terms for the updated term and change all necessary term_taxonomy associations. term_taxonomy_id will stay the same, but term_id will change. This is a case of a shared term being split into separate terms.

In most cases, term splitting will go unnoticed. However, there are some plugins and themes that store term IDs as static data. In these cases, a changed term ID has the potential to cause various sorts of problems.

Who is affected?

In brief, any plugin that independently stores term IDs in the database – postmeta, usermeta, the options table, etc. @mboynes and I looked over the top 100 most popular plugins on wordpress.org, and found eleven that will need changes in order to anticipate term splitting: Jetpack, WordPress SEO by Yoast, Google XML Sitemaps, All in One SEO Pack, Mailpoet, Advanced Custom Fields, Ninja Forms, Types, Custom Sidebars, Paid Memberships Pro, WordPress Download Manager. If you’ve written a plugin, theme, or other customization that stores term IDs in a static way, you’ll be affected too.

What steps should you take?

WP 4.2 will include a number of tools that developers can use for split term ID migration. The most important is the 'split_shared_term' action. Here’s a concrete example, taken from Jetpack. Jetpack stores a post_tag ID in the ‘featured-content’ option, and in some cases, the tag’s term ID may be shared with another taxonomy term. The following is a simple function that could be used to detect when the term is split, and to update the option accordingly:

function jetpack_update_featured_content_for_split_terms( $old_term_id, $new_term_id, $term_taxonomy_id, $taxonomy ) {
    $fc_settings = get_option( 'featured-content', array() );

    // Check to see whether the stored tag ID is the one that's just been split.
    if ( isset( $fc_settings['tag-id'] ) && $old_term_id == $fc_settings['tag-id'] && 'post_tag' == $taxonomy ) {
        // We have a match, so we swap out the old tag ID for the new one and resave the option.
        $fc_settings['tag-id'] = $new_term_id;
        update_option( 'featured-content', $fc_settings );
    }
}
add_action( 'split_shared_term', 'jetpack_update_featured_content_for_split_terms', 10, 4 );

In addition to the 'split_shared_term' action, 4.2 will store information about all terms that are split. Developers can access this information after the fact by using wp_get_split_term( $old_term_id, $taxonomy ). For more details on 'split_shared_term' and wp_get_split_term(), including a number of practical examples, be sure to check out the page in the Plugin Developer Handbook.

What will happen if plugins and themes don’t update?

The vast majority of terms are not shared between taxonomies; shared terms themselves are an odd edge case. The vast majority of plugins and themes do not store term IDs. And, moreover, WP 4.2 will only split terms when a shared term is updated (eg, when its name is updated in the Dashboard) – an infrequent event. (Though note that there are plans to force all shared terms to be split in a future release.) Even in cases where a plugin is technically affected, depending on the way the plugin uses its stored term IDs, there may be no perceptible effect at all.

That said, there are cases where failure to account for split terms could result in fairly disruptive behavior. In the case of Jetpack, for example, Featured Content would stop displaying correctly if the term split were not caught properly. For this reason, it’s critical for developers to examine the way they store term IDs, and to update their plugins and themes accordingly, in time for 4.2 in April.

Questions or concerns? Read over the practical guide in the Plugin Developer Handbook, or leave a comment below.

#4-2, #taxonomy

Emoji Chat Meeting Notes, February 12, 2015

The full meeting archive is available here.

1. Why we’re doing this

So, here’s a bit of back story.

As of r31349, WordPress partially supports emoji. ~60% of WordPress sites are running MySQL 5.5 or later (so can be upgraded to store emoji), and ~40% of browsers natively support emoji. Emoji are a wildly popular method of communication, so we can expect them to be heavily used as soon an they’re available. The problem is, 60%/40% means a really bad experience for a huge number of our users, who’ll try to use emoji, and fail.

This is where the emoji feature plugin comes in to play. In order to help the 40% of WordPress sites that can’t be upgraded to store emoji natively, the wp_encode_emoji() function will turn them into HTML entities. Due to the unimaginable joy that character sets brings me, this will only be applied to sites using the utf8 character set, which accounts for the vast majority of WordPress sites – utf8 has been the default character set since r4860.

To help the 60% of browsers that don’t display emoji natively, we’re using the Twemoji image set as a fallback. This lets us show emoji everywhere, without causing extra load where emoji are already supported, mobile browsers being the important example here.

Now, there have been some concerns brought up previously that I’d like to address.

“Is this really appropriate for core?”

Yes. (Obviously that’s my answer, or we wouldn’t be here.) WordPress is is the business of making communication simple and accessible for all. Tech users everywhere have clearly chosen emoji as a means of communication, so it’s up to us to make sure they can do that within WordPress as easily as possible, or risk being left behind.

“Should we be concerned about changing the images in the future? Wouldn’t we be altering users’ content?”

No. By using Twemoji only when we can’t provide native support for emoji, it’s a pretty clear message that while the general appearance of emoji stays the same, the actual sprite used can differ significantly between platforms. (For example, every emoji set except Android uses a left hand for :thumbsup:.) As more browsers add native support for emoji, Twemoji usage will drop, reducing even further any impact we can have on users.

And so, that brings us to today.

2. The current state of the plugin

The plugin is very close to done. The editor plugin needs some attention, which @azaozz will be providing soon. There are a few bugs to discuss, which are mostly around fallback behaviour in browsers that don’t support emoji natively. Apart from that, the basic functionality is pretty much how I would expect it to appear in core. It’s had a brief review from the accessibility team, with only some minor alterations needed. The Twemoji images won’t be included in wordpress.zip, as it’s a total of 3.4MB of images. They’re currently hosted on WP.com’s CDN, but we’re investigating other options for where to host them, probably the W.org CDN. Given that the wp-admin Dashboard also loads things from Google, I have no problem with hosting them on an external CDN. There will naturally be a filter on the URL, to allow local hosting for sites that don’t want to use the CDN.

One of the major concerns at the moment is that we’re going to be splitting data formats, depending on if the site uses the utf8mb4 or the utf8 character set. utf8mb4 stores emoji natively, while utf8 requires us to HTML encode the emoji characters. In the futures, we’ll look at upgrading sites to utf8mb4 if they’ve upgraded their MySQL since WordPress 4.2, but that leaves the potential for mixed encoding – old posts having HTML encoding, new posts having the native characters. A post will be automatically updated to native upon saving, but do we need to consider upgrade routines, to go through all old posts and convert them?

Export/import also needs thorough testing, particularly when importing and exporting between sites having different character sets.

3. Unicode 8.0: the future of emoji

To talk about the future of emoji, you need to know a little bit of history. At the basic level, emoji are all single characters defined within the Unicode standard. However, they also support modifiers. Modifiers are a second character following the first, which usually causes the two characters to be merged into a single character when rendered. A good example of this is flag emoji.

The character G is U+1F1EC. The character B is U+1F1E7 (these characters are different to their ASCII equivalent). When used individually, they’ll display as that letter. When combined next to each other, they’ll display as the British flag.

So, Unicode 8.0 will two interesting things: a set of 37 new emoji, and skin tone modifiers. When a skin tone modifier character is placed after any face or person emoji, the emoji will show with that skin tone. Unicode 8.0 is due to be finalised in August 2015, so we and (Twemoji) will be looking at adding support for these then.

From a technical perspective, it just means we need to be aware that emoji are not always one character, and the methods for detecting multi-character emoji are about to get more complex.

We’ll also be able to detect if a browser is able to render the new emoji and skin tones, and fall back to Twemoji if they can’t. I don’t have a timeline for when browsers will support the new emoji, so I think it’d be good for us to get ahead of the curve then.

utf8mb4 stores anything in the Unicode address space, including unallocated characters, so I don’t expect any problems with storage of new emoji.

#emoji, #x1f4a9

Emoji Chat Agenda, February 12, 2015

Here’s the agenda for Thursday’s Emoji Chat in the #core channel on Slack.

Time/Date: February 12 2015 23:00 UTC

  1. Why we’re doing this – @pento
  2. The current state of the plugin – @pento
  3. Unicode 8.0, and future plans – @pento
  4. Open Floor/ranting about the evils of emoji – everyone

See you tomorrow!

#emoji, #x1f4a9

Press This User Testing 2 10 I ran…

Press This User Testing 2/10

I ran three folks through User Testing the Press This Plugin as it stood on 2/9/2015 – the big takeaways as I see them:

  • The bookmarklet installation isn’t as easy as it could be: at least two of the testers struggled to get it into place correctly, instead dragging the icon to their address bar. They also both took a bit to recognize the draggable icon as something they could interact with, though the hover state helped here.
  • Once the bookmarklet was in place, each tester was able to use it without trouble, though there was a little confusion re: the image grid v. the camera icon.

Their answers to the prompted questions:

What did you like about the Press This bookmarklet?
I can use it on every website I am with just one click.
Did you miss any content creation tools, like post format, tags, or categories?
I find it satisfing.
At any time during this test, did you not feel safe, smart, or powerful? Can you tell us about that?
Yes, at the time when I could not find Press this in WordPress. I would preffer if there were some more clear indicators for it.
How likely would you be to use a tool like Press This, on scale of 1 – 10? Why would you rate it the way that you did?
Probably 9, because it is cleverly made and I like it, but I would give 10 after some time of using when I am more confident about it.

What did you like about the Press This bookmarklet?
It makes it easier to repost articals on my own website.
Did you miss any content creation tools, like post format, tags, or categories?
No I did not.
At any time during this test, did you not feel safe, smart, or powerful? Can you tell us about that?
I felt safe the whole way, but did not feel smart or powerful the whole time. I got confused when I needed to “install” the bookmarklet shortcut I would call it. I thought that I would need to download and install an app so that made me confused when I just needed to save it.
How likely would you be to use a tool like Press This, on scale of 1 – 10? Why would you rate it the way that you did?
I would say an 8. Because it seems easy to use, the instructions on how to use it is just a bit confusing.

What did you like about the Press This bookmarklet?
I really like the idea behind this to just freely create our own pages on our own and after I figured out how to use the bookmarklet, it could be quite easy to use.
Did you miss any content creation tools, like post format, tags, or categories?
I think the costumization part of the site was pretty well made and it has a lot of possibilities to create the site as I like it, so I don’t really miss anything else.
At any time during this test, did you not feel safe, smart, or powerful? Can you tell us about that?
At the beginning this bookmarklet idea was a bit frightening, because I thought that this would effect my browser too and for the correct use, it would put something on my computer but as soon as I found out the mechanism of this bookmarklet, there wasn’t any other problem with safety. Actually I didn’t feel really smart during this test, because the process for using this bookmarklet was a little bit confusing and I didn’t encountered a usage like this before for a bookmarklet, so it was pretty hard for me to figure out the mechanism. There wasn’t any problem with the powerful part, because it is really simple just to copy an article and use it as I like it, so that felt very good.
How likely would you be to use a tool like Press This, on scale of 1 – 10? Why would you rate it the way that you did?
I would give it an 8, because when I would like to create my own articles and press about the everyday happenings, then this could be very helpful to just copy the part I want in a very simple way and then I can customize it anyway I like with a very well made site. The reason it is not a 10 is because I think it is a bit hard to get the hang of this program and find the way to make it work, like find the proper parts on the webpage, but after some practise I think this shouldn’t be a problem.

You can watch all three videos here:
https://drive.google.com/folderview?id=0BwIP25_eL_Jca1BpRjZnSFU3QlE&usp=sharing

#press-this

Press This update 2/10

The merge window is supposed to open tomorrow for feature plugins. That means it’s crunch time. Here’s where the new Press This is.

Feature Parity

one of the requirements of core is general feature parity with the old version of Press This. Here’s a comparison chart of where the new Press This is.

Feature Old New
Drag & drop install on desktop Yes Yes
Editor, including: title, image/gallery addition Yes Yes
TinyMCE buttons (minus kitchen sink) Yes Mostly [1]
Ability to publish or save as draft Yes Yes
Post formats Yes Yes
Categories Yes In Progress
Tags Yes In Progress
Content Scraping Yes Improved [2]
Media Discovery Yes Improved [3]
Alert before closing/navigating away Yes Todo
Can add to bookmarks Yes Yes
Responsive, clean design, updated icons No Yes
Fast loading (speedy!) No Yes
Mobile installation No Yes

[1] A number of TinyMCE buttons are removed intentionally. Only necessary WYSIWYG buttons are shown now.
[2] Not only is it included, but it’s quite a bit smarter than the previous one.
[3] Now is actually quite exposed in the UI.

Before Core Merge

Prior to merge, there’s a bunch that still needs to be done. With that in mind, @DrewAPicture has given us an extra week to accomplish this. Even still, we have a list of things that need to be done prior to devchat tomorrow, with the rest of the list done in a week.

Before Tomorrow’s Devchat (February 11)

Before Next Wednesday’s Devchat (February 18)

  • Categories functional and live
  • Install flow functional and live
  • Side navigation JS improvements (functional)
  • Flows posted to make/flow comparing the old and new design (@ryan)
  • Another round of accessibility testing (when UI is finished)
  • All accessibility concerns addressed
  • Categories functional and live
  • JS inline docs
  • HTML validation
  • Security audit
  • Usability testing reviewed and any issues addressed
  • Verify localization (including rtl) is functional
  • Browser testing on all browsers (support parity with old Press This plus mobile)
  • Unit tests?

If we’re able to accomplish all of the above, we should be ready for merge on February 18.

Daily Chats

In this final rundown, let’s meet daily in #feature-pressthis at 17:00 UTC to make sure we’re on track for merge. Anyone interested in helping, please join us.

All development is done on Github: https://github.com/MichaelArestad/Press-This/

Plugin on the plugin repo: https://wordpress.org/plugins/press-this/

#press-this

New chapters for Ryan and Westi

WordPress lead developers Ryan Boren (@ryan) and Peter Westwood (@westi) started contributing to WordPress more than a decade ago. Ryan and Peter, along with Mark and Matt, served as the foundation for much of the early years.

For some time now, Ryan and Peter have avoided weighing in on technical matters. Very simply, when you aren’t able to be active in development, you know you’re not up to speed, and you realize your words shouldn’t carry the weight that they do. Being able to make this judgment is one of the things that makes both of them such great leaders.

We’ve all been there, at least for particular features or releases. It’s worth noting, for example, that my own time on core has been cyclical for years, as sometimes I end up working full time on the security team, maintenance releases, the WordPress.org site, or related projects.

The great thing is, there are a lot of fantastic developers who have stepped up over the last few years to seamlessly fill in the huge holes they’ve left. Some of that culminated in promoting Helen and Dion to lead developer yesterday, and my own promotion three years ago.

When I started contributing, I received a lot of advice and learned a lot from both of them. Peter reviewed a lot of my code and was the guy who would revert my code when I broke something. 🙂 Ryan became my mentor and pushed me to become the engineer I am today.

And so, it is with mixed emotion I share that Ryan and Peter have stepped down as lead developers.

Peter will be moving into a dormant/inactive/emeritus status. We hope to have him back when his life and work allows. In the meantime, you may see him committing a bug fix here and there, as he is wont to do.

Ryan has been focusing all of his energy on improving UX for more than a year, especially for mobile and touch devices, and especially for workflows like media management. So I’m pleased to say he’ll continue to do that: Ryan will be spearheading UX for WordPress in 2015. It’s been a while since we’ve had someone truly focusing on just UX, so this is really exciting.

Along with yesterday’s announcement, the active lead developers are @markjaquith, me, @azaozz, @helen, and @dd32.

Please join me in congratulating Ryan and Peter on an epic run. 🙂

New lead developers: Helen and Dion

I’m pleased to announce that Dion Hulse (@dd32) and Helen Hou-Sandí (@helen) are now lead developers of the WordPress project.

The two are already highly respected leaders in the community. Dion has architected some of the most important code in WordPress over the years. He started by building the plugin updater way back in 2007, helped lay the foundations of custom post types and taxonomies, and more recently, designed and implemented automatic updates. He has essentially been one of core’s main architects for years, all while providing advice, code review to newcomers (myself included, long ago).

Helen’s tremendous impact on the project is surely known to all of you. She champions the project’s philosophies, she’s been a key leader on dozens of features large and small, including the media UI, CSS architecture, and the jam-packed WordPress 4.0 release. Her strong engineering skills are backed by her natural leadership, sound judgement for user experience design, and the mentorship of countless contributors.

Please join me in congratulating Helen and Dion!

Dev Chat Summary, December 10th

https://wordpress.slack.com/archives/core/p1418245350000526

https://make.wordpress.org/core/2014/12/10/release-candidate-and-todays-dev-meeting-agenda/

Misses

  • RC

Decisions

  • About page text will land tonight (Wednesday the 10th).
  • Moving API docs into the plugin and theme handbooks will be discussed at the next docs meeting.
  • A heads up email to plugin devs will go out after RC.
  • RC will go out tonight.
  • @nacin will alert hosts and one-click installers in advance release.
  • 4.1 final targeted for Tuesday.
  • Release dry-run on Monday.
  • Status check-in meeting on Sunday.

Assignments

  • @stephdau and @nacin will enable localized results for plugins after RC.
  • @nacin will work on the plugin developer email.
  • @melchoyce @ryelle @helen @jorbin will work on the about page. First draft design due by Friday.
  • @nacin @markjaquith will make the final decision on Focus, Distraction Free Writing naming, branding, text.
  • @nacin will alert hosts and one-click installers in advance of release.

Links Mentioned

https://wordpress.org/support/forum/alphabeta
https://core.trac.wordpress.org/ticket/30337
https://wordpress.org/about/roadmap/
https://core.trac.wordpress.org/query?status=!closed&version=trunk&order=modified
https://core.trac.wordpress.org/query?status=!closed&version=trunk&order=modified&milestone=Awaiting+Review&group=type

Continue reading

If you’ve written a child theme for Twenty…

If you’ve written a child theme for Twenty Fifteen, please note that some of the new pagination functions have been renamed for a bit of clarity:

  • the_pagination() is now the_posts_pagination()
  • get_the_pagination() is now get_the_posts_pagination()
  • These two functions now emit a “posts-navigation” HTML class, instead of “paging-navigation”

@obenland‘s original post on new template tags in WordPress 4.1 has been updated with these changes.

#4-1, #bundled-theme, #dev-notes, #twentyfifteen

Open Update Thread, Week of December 8th

What’s happening?

#open-update-thread