Dev Chat Summary: September 6th (4.9 week 6)

This post summarizes the dev chat meeting from September 6th (agendaSlack archive).

4.9 schedule review

  • 3 weeks until the feature project merge deadline, 4 weeks until Beta 1
  • Customizer improvements for merging Changeset drafting and scheduling has yet to kick off development, designs are nearing completion (see: #39896 and #28721)
  • Gallery widget is still under development but it seems to have stalled, TODO’s noted on related GitHub PR
    • @joemcgill to look into avoiding serializing attachments data in the widget this week
  • @obenland working on wrestling the widget mapping issue when switching themes (see: #39693)
  • Page on Front progressing slowly, likely not ready for dev before Feature Merge
  • Theme switching issue for nav menu mapping has already been merged in trunk (see: #39692)
  • CodeMirror feature plugin (aka Better Code Editing) needs testing and a few outstanding issues that would benefit from contributors. Plan is to merge this week.
  • @psykro to look into #9757
  • “Add Media” button in the Text widget great opportunity for new contributors
  • #35827 could use an owner and remaining items in 4.9 Goals post could use contributors to help land in the release

Editor update

Iterating in trunk

  • @matt: I’m fine with more iteration happening in trunk vs how we’re bouncing patches around Trac so much
  • @matt: I’m okay with parts of trunk being broken as we iterate in this phase of dev
  • @desrosj: Do we have an established process for reverting things that break?
  • @obenland: I think we’re not talking about “PHP fatals”-broken, but rather a feature maybe not fully functional

HTML5 input types for validation

  • @afercia: any thoughts about relying on HTML5 input types browsers built-in validation only?
  • @azaozz: used to be buggy, seems to be working properly now
  • @afercia: seems to me still premature to rely on required for validation
  • @afercia: looking to leads to make a decision as new browsers support policy
  • @asaozz: Worth some testing, especially on the “lower end”, IE11
  • @afercia: there are still CSS rules in ie.css for Internet Explorer 6 (and 7, and 8). Can they just be dropped?
  • @azaozz: no need of ie.css in my honest opinion
  • @azaozz: intention is not to completely break old browsers if they still work, but to stop testing in them
  • @clorith: concerned about users locked into older browsers, like IE8, and keeping option for them to enqueue scripts relevant to their browser
  • @afercia: I wanted to start the discussion about this as it relates to the new browsers support policy

General announcements

#4-9, #core, #core-customize, #core-editor, #dev-chat, #gallery, #gutenberg, #html5, #summary, #trunk, #widgets

Fixes to Text widget and introduction of Custom HTML widget in 4.8.1

The 4.8 release caused issues for many sites that had custom HTML in Text widgets, which until now had been common practice. So we’ve been working hard on fixes in the 4.8.1 release which aim to simultaneously serve the needs of novice users and advanced users alike: the rich Text widget (introduced in 4.8), a legacy mode for the Text widget, and a Custom HTML widget.

For more background on the changes in 4.8, see Addition of TinyMCE to the Text Widget. To review, the Text widget in 4.8 includes TinyMCE—the same visual editor used for writing post content—and it looks like:

Text Widget Legacy Mode

The issues with the introduction of TinyMCE to the Text widget revolve around the ways that TinyMCE attempts to clean up HTML code by deleting empty elements (such as those for dashicons) and dropping attributes it may not recognize (such as HTML5 Microdata attributes). Also with the 4.8’s removal of the “automatically add paragraphs” checkbox, there were also issues related to paragraphs and line breaks being added incorrectly.

Note that the Text widget was already designed to preserve the old behavior of the widget until it was modified and thus upgraded, so there are many instances of Text widgets in the wild today that could very well begin to break upon being modified. For this reason the issues were not reported right away and instead started to trickle in steadily after the release.

There were various solutions that were considered, but the one that had the consensus among contributors was:

[Check if the Text widget] was previously saved from an older version of WordPress before TinyMCE was added to the Text widget. If it is such a pre-existing Text widget instance, then use heuristics to detect if TinyMCE would negatively impact the contents of the widget, including the auto-p checkbox being unchecked, whether there are empty tags, and whether there are spandivscript, or style tags. When the Text widget is in this legacy mode, it can have a notice that informs users of the new HTML Code widget and that it should be used going forward. Likewise, in the new mode when TinyMCE is present, when the Text (HTML) tab is selected, there can be a note (perhaps an admin pointer) that encourages users to use the HTML Code widget instead. By implementing this, novice users with basic content in their widgets win, and advanced users with custom HTML content in their widgets will cease from being negatively impacted.

The Text widget in legacy mode looks the same as the Text widget before 4.8, but with the addition of a new notice:

The legacy mode will only be presented for widgets created prior to 4.8.0 that have instance data which match the logic in the WP_Widget_Text::is_legacy_instance() method. The legacy mode will not be presented to newly created Text widgets. Once a Text widget is opened and saved in legacy mode, it will permanently stay in legacy mode. There is a new instance property called “visual” which will be set to false when a widget is saved in legacy mode. When a new Text widget is created, it is opened in the default visual mode and the new instance will get saved with visual=true.

Text Widget Filters

There is a change in how the filter instance property was used in 4.8.0: in that release, when a Text widget was modified, the fact that it had been upgraded was stored by overloading the filter boolean property to also have the value of "content", indicating that the widget gets content filters applied like a post does. Since this string is a truthy value, I reasoned it would normally work the same in filters that check ! empty( $instance['filter'] ), but it would fail in cases where a plugin tried true === $instance['filter']. So 4.8.1 reverts the overloading of the filter property to again be a boolean, and this should improve compatibility for widget_text filters. Whenever a Text widget is modified with the default visual mode (with TinyMCE) it will get both visual=true and filter=true saved in its instance. When a Text widget is modified in the legacy mode, it will always get visual=false and its filter property will reflect the checked state of the auto-paragraph checkbox.

Another note on filters: special consideration was made for shortcodes in the Text widget given the frequency of plugins and themes adding shortcode support (since the widget does not recognized them by default in core). Plugins and themes have done add_filter( 'widget_text', 'do_shortcode' ) to add support. Since the widget_text filter applies before the new widget_text_content filter (as of 4.8), it will apply before wpautop will have applied, resulting in the possibility of extra line breaks being added undesirably if the shortcode output has new line characters. So to help prevent that from happening, the Text widget will temporarily move the do_shortcode handler from widget_text to widget_text_content just in time while the filters are being applied. See the relevant logic.

Help Pointers

For users who are accustomed to pasting HTML into the Text widget, when an attempt is made to paste markup into the visual editor a pointer will be displayed informing them that they should paste it into the Text tab instead, or to alternatively use the new Custom HTML widget (see section below):

Likewise, when a user opens the Text tab, it will also open a pointer to inform them of the Custom HTML widget:

While pointers are normally displayed on upgrades, these pointers will be displayed even on new installs since they reflect changes to long-standing behavior for the Text widget that users have become accustomed to. Any tutorials that instruct users to use the Text widget for pasting in arbitrary HTML should be updated to instruct the users to select the Custom HTML widget instead.

Custom HTML Widget

For advanced users or for any use case where arbitrary HTML needs to be displayed in a widget (such as a signup form or a 3rd party JavaScript widget), there is now a dedicated “Custom HTML” widget that is specifically for this purpose. It looks very similar to the classic Text widget, except it has a monospace font and it lacks the auto-paragraph checkbox:

Since users are prompted (per the pointers above) to try using the Custom HTML widget instead of the Text widget for some use cases, it is important that the widget content be able to be freely copied between the Text widget and the Custom HTML widget. For this reason, the Custom HTML widget retains the application of the widget_text filters like the Text widget does. The type of widget for which the filter is applying can be determined by looking at the type of the WP_Widget instance being passed as the last filter argument. When the widget_text filter is applied, it will pass the second $instance parameter in the same format as the Text widget, with title, text (instead of content), and filter and visual properties that are always both set to false (as if the instance was in legacy non-visual mode). In addition to re-applying the widget_text filter, the Custom HTML widget has a dedicated widget_custom_html_content filter whereas the the Text widget has a dedicated widget_text_content filter.

In addition to filter compatibility, the Custom HTML widget also tries to retain theme styling compatibility by using the same widget_text CSS class name on the outer widget wrapper and textwidget on the inner wrapper around the content itself. For any themes that wish to style the Custom HTML widget alone, there are the widget_custom_html and custom-html-widget class names used on the outer and inner wrapper elements respectively. For themes that wish to style the Text widget alone and exclude the Custom HTML widget, the :not() pseudo selector can be used, for example .widget_text:not(.widget_custom_html) and .textwidget:not(.custom-html-widget) for the outer and inner wrappers, respectively.

The markup generated by a Custom HTML widget on the frontend will look like:

<section id="custom_html-6" class="widget_text widget widget_custom_html">
  <h2 class="widget-title">My Title</h2>
  <div class="textwidget custom-html-widget">My Content</div>
</section>

This same Custom HTML widget’s instance data will look like:

{
  "title": "My Title",
  "content": "My Content"
}

For more specifics on the Custom HTML widget, refer to the subclass: WP_Widget_Custom_HTML.

Here is a list of tickets related to the Text widget and Custom HTML widget which are closed in the 4.8.1 release:

  • #40907: Introduce widget dedicated for HTML code
  • #40951: New Text Widget – Switching Between Visual/Text Editor Strips Out Code
  • #40960: Set `’filter’ => ‘content’` on starter content “business info” widget
  • #40960: Widgets: The Text widget should respect the “Disable the visual editor when writing” setting
  • #40972: TinyMCE editor in Text widget does not have RTL contents
  • #40974: Updated text widget do not save text (when using paste)
  • #40986: Widgets: text widget and media widgets cannot be edited in accessibility mode
  • #41021: Text widget does not show Title field or TinyMCE editor
  • #41158:  Increase tinymce panel z-index
  • #41361: Text widget can raise JS error if customize-base is enqueued on widgets admin screen
  • #41386: Text Widget – Wording – Legacy Mode 4.8.1 beta
  • #41392: Theme styles for Text widget do not apply to Custom HTML widget
  • #41394: Text widget: Rename legacy mode to visual mode and improve back-compat for widget_text filters

 

#4-8-1, #dev-notes, #tinymce, #widgets

Dev Chat Summary: July 12th (4.8.1 week 4)

This post summarizes the dev chat meeting from July 12th (agendaSlack archive).

4.8.1 planning

  • @westonruter to commit #40951: Add legacy mode to Text widget to trunk, all trunk commits need to be merged into the 4.8 branch
  • Custom HTML widget needs to be amended to an existing file to be part of the minor release, agreement to utilize wp-includes/default-widgets.php solely for the benefit of 4.8.x
  • Testing, especially #40907 and #40951 (testing steps)
  • Beta – Monday, July 17th (soft string freeze, translations begin on about a dozen strings)
  • RC – Monday, July 24th (hard string freeze)
  • Launch – Tuesday, August 1st (🎉)

Editor / Gutenberg

General announcements

#4-8, #4-8-1, #core, #core-editor, #dev-chat, #feature-plugins, #summary, #widgets

Addition of TinyMCE to the Text Widget

Note: Update: A follow-up post has been published regarding Fixes to Text widget and introduction of Custom HTML widget in 4.8.1.

In its first couple years, WordPress lacked rich/visual text editing. Before TinyMCE was incorporated in WordPress 2.0, users had to edit post content as raw HTML with some support from the Quicktags buttons. When widgets were introduced in WordPress 2.2, the Text widget was included which allowed a user to add content to their sidebar. Nevertheless, unlike the post editor, the Text widget did not incorporate TinyMCE, nor did it include Quicktags. For twelve years, since TinyMCE was added to core in 2005, users have had to hack around with HTML in their Text widgets to do things as simple as make text bold or add links. This has been featured even as recently as the 4.7 release video. Well, as of WordPress 4.8, the Text widget is finally getting the same treatment as the post editor with the introduction of TinyMCE for visual text editing, while still supporting raw HTML editing via a Text tab but now with the additional help of Quicktags:

Text widget: visual tab Text widget: text (HTML) tab

A primary reason for the long delay in incorporating TinyMCE into the Text widget was the difficulty of cleanly instantiating another copy of the WordPress visual editor dynamically after the page has loaded. Since WordPress 3.3 there has been the wp_editor() PHP function for instantiating an editor for the initial page load, but there was no facility for instantiating editors afterward, such as when adding a new Text widget to a sidebar. So in #35760 a new JS API was introduced for dynamically instantiating WP editors on the page: wp.editor.initialize(). This JS API is used for the new TinyMCE-powered Text widget; for more, please see Editor API changes in 4.8.

Note that by default the Text widget only features buttons for bold, italic, unordered list, ordered list, and link. If you want to add additional buttons to the toolbar, you may use a plugin to enqueue the following JS to add the blockquote button, for example:

jQuery( document ).on( 'tinymce-editor-setup', function( event, editor ) {
	if ( editor.settings.toolbar1 && -1 === editor.settings.toolbar1.indexOf( 'blockquote' ) ) {
		editor.settings.toolbar1 += ',blockquote';
	}
});

Likewise, a custom button can be added via code like:

jQuery( document ).on( 'tinymce-editor-setup', function( event, editor ) {
	editor.settings.toolbar1 += ',mybutton';
	editor.addButton( 'mybutton', {
		text: 'My button',
		icon: false,
		onclick: function () {
			editor.insertContent( 'Text from my button' );
		}
	});
});

A reason for why there is no “Add Media” button for the editor in the Text widget is that WordPress 4.8 also includes dedicated media widgets for adding images, video, and audio to sidebars. Another reason is a current technical limitation whereby arbitrary media embeds (oEmbeds) fail to work properly outside the context of post content (see #34115). For this reason if you try to embed a video or something else by pasting in a URL it will not currently expand into the embedded content. The same is true for shortcodes: they are not processed by default since many shortcodes presuppose a global $post as the context within which they expect to be running. Plugins may opt-in selectively to support individual shortcodes by filtering widget_text as they have had to do for many years now.

While not supporting shortcodes, the updated Text widget does have some of the same filters applying to it as the_content, in particular:

  • wpautop
  • wptexturize
  • capital_P_dangit
  • convert_smilies

The Text widget has supported a “filter” instance property which was reflected in the UI via a checkbox for whether or not to automatically add paragraphs and line break tags (wpautop). For the updated Text widget, this checkbox is removed entirely in favor of aligning its behavior with the post editor where this behavior is always enabled. These filters only apply on Text widgets that have been updated/touched after the 4.8 upgrade. When a Text widget is modified in 4.8, the filter instance prop gets set to a static string “content both the filter instance property and a new visual property will be set to true and then it will apply a new widget_text_content filter which then apply the above functions to the widget text. For Text widgets created prior to 4.8, they will be opened in a legacy mode with the old behavior when it is determined that TinyMCE could mutate the content undesirably; such legacy instances will be saved with visual=false instance property. Read more about Fixes to Text widget and introduction of Custom HTML widget in 4.8.1.

Important: When pasting HTML into the “Text” (HTML) tab of the Text widget, any extraneous line breaks should be removed or else unwanted paragraphs and line beaks may result. This is particularly important when you paste in script or style tags (as in the case of 3rd-party JavaScript embeds), since auto-inserted paragraphs will cause script errors; this will be fixed in #2833. This behavior aligns with longstanding behavior in the post editor, so it is not new, although it does differ from how the Text widget has previously behaved. As noted above, for previously-existing Text widgets that had the “auto-add paragraphs” checkbox unchecked (and thus the filter instance prop set to false), the previous behavior of not doing wpautop will be maintained: only once the widgets are modified will any extraneous line breaks need to be removed. Any such Text widgets created prior to 4.8 will be go into a legacy mode as of 4.8.1 to prevent any data loss. Going forward, arbitrary HTML should be placed into the new Custom HTML widget instead.

The incorporation of TinyMCE into the Text widget has necessitated constructing the widget’s form fields differently than how they are normally done. Widgets in core have historically utilized static HTML for their control form fields. Every time a user hits “Save” the form fields get sent in an Ajax request which passes them to the WP_Widget::update() method and then the Ajax response sends back the output of WP_Widget::form() which then replaces the entire form. (Note widgets in the Customizer behave differently since there is no Save button in the widget, as updates are synced and previewed as changes are made; read more about Live Widget Previews.) This worked for static HTML forms in the past, but TinyMCE is a JavaScript component. To avoid having to rebuild TinyMCE every time the user hits Save on the admin screen, the Text widget puts the title field and TinyMCE text field outside of the container that is “managed” by the server which gets replaced with each save. (A similar approach has also been employed by the new media widgets in 4.8.) The fields in the Text widget’s UI sync with hidden inputs which get synchronized with the server; in the Customizer, changes to the TinyMCE field will be previewed after 1 second with denouncing. The container for the Text widget’s fields is .text-widget-fields and the traditional container for a widget’s input fields as rendered by WP_Widget::form() is .widget-content:

Themes should already account for the most common HTML elements within the text widgets and provide appropriate styles for them—the addition of the editor toolbar increases the likelihood its elements will be used in the future. Most default themes needed additional styles for ordered and unordered lists within widgets, so theme authors are encouraged to double-check their themes and test them with content in the text widget that includes markup provided by the editor toolbar.

Initial groundwork for shimming JavaScript into widgets was added in 3.9 via the widget-added and widget-updated events. A more recent proposal for making JavaScript more of a first class citizen in widgets can be found in #33507 and the media widgets incorporate some of its patterns that were also prototyped in the JS Widgets plugin. The synchronization of a widget’s state (instance properties) via hidden text fields can be eliminated once a widget’s state can be fully represented in a JavaScript model.

The Text widget is implemented as a Backbone.js view which is available at wp.textWidgets.TextWidgetControl. Instances of this view are then stored in the wp.textWidgets.widgetControls object. The widget’s control JS will only instantiate for a given widget once it is first expanded, when the widget-added event fires. What’s more is that the widget will only first initialize once the container is fully expanded since TinyMCE is not able to initialize properly inside of a container that is animating. In order to capture when TinyMCE inside the Text widget is initialized, you should use a TinyMCE event like tinymce-editor-setup. Note also that due the Document Object Model, when a widget is moved to a new location in a sidebar and this the TinyMCE iframe is moved in the DOM, this dynamically-created iframe is reloaded and thus emptied out. For this reason, every time a Text widget is moved to a new location the TinyMCE editor in the widget will be removed and then re-initialized. Keep this in mind when extending.

Keep also in mind that the Text widget will likely undergo many more changes with the incorporation of the Gutenberg editor, and that widgets themselves will likely see many changes to align with Gutenberg’s editor blocks which are now being prototyped.

#4-8, #dev-notes, #editor, #widgets

Pre-instantiated Widget Registration in 4.6

Since WP_Widget was introduced in 2.8 the register_widget() and unregister_widget() functions required the class name (string) of a WP_Widget subclass to be supplied. As of 4.6 these functions also accept a class instance (object) of a WP_Widget subclass as well. See #28216.

Two key benefits for allowing objects to be instantiated are:

  1. Widgets can now be instantiated and registered with constructor dependency injection.
  2. New widget types can now be added dynamically, such as adding a Recent Posts widget for each post type, per #35990.

#4-6, #dev-notes, #widgets

HTML5 Widgets in WordPress 4.2

Note: HTML5 widgets theme support in 4.2 has been reverted pending a decision about the correct container to use

With the upcoming WordPress 4.2 release, widgets have been added to the growing list of HTML5 supported markup. Once you’ve added HTML5 support for widgets, WordPress will use the <aside> element, instead of the generic list markup.

To declare that your theme supports HTML5 widgets, add the following call to your theme’s functions.php file, preferably in a callback to the after_setup_theme action:

add_theme_support( 'html5', array( 'widgets' ) );

For forward compatibility reasons, the second argument cannot be omitted when registering support. Otherwise, a theme would automatically declare its support for HTML5 features that might be added in the future, possibly breaking it visually.

If there are any questions about the current implementation, feel free to leave a comment below.

#4-2, #html5, #themes, #widgets

Live Widget Previews: Widget Management in the Customizer in WordPress 3.9

The WordPress 3.9 release includes widget management in the Customizer: previewing changes to widget areas before publishing them live for the world to see. This feature was birthed out of the Widget Customizer plugin which started development in the 3.8 release cycle, it was formally proposed for merging into core at the start of the 3.9 release cycle, and then it was merged and followed by many changes. I wanted to follow-up here to talk through the changes to Widget Customizer since it was initially proposed (read this post for a full overview of the feature). Here is what has changed since then:

No more opt-in for theme support

The Widget Customizer plugin implemented a mechanism for doing fast live widget previews. By default, all changes made to settings in the customizer cause the preview window to do a full page refresh. This causes a lag between when you make a change and when the change appears. The customizer also supports an alternate non-refresh mechanism for previewing changes in the customizer (the postMessage transport) which you often see themes implement for live-previewing the blog name or the header color.

Implementing such immediate live-previews requires all of the logic for previewing the change to be implemented in JavaScript, in addition to the underlying PHP logic for rendering the change when the setting is saved. This is not possible for widgets, since they all use PHP to handle the sanitization of the form inputs and for rendering arbitrary content that a widget may contain. (You’ll notice that when you make a change to a widget in the customizer, it actually does an Ajax request to submit the widget form for sanitization every time.) Well, to get around having the customizer preview refresh for every widget change, we implemented an Ajax renderer for the widgets, and then used the postMessage transport to initiate this widget render request, and then the rendered widget response would get swapped out in the DOM.

Long story short, we decided to remove this feature to use postMessage for faster live previews. We have created #27355 to re-add this feature, but to generalize it so that any customizer control can take advantage of doing these partial preview refreshes. So for now, you’ll have to wait a moment for the preview to refresh when you make a change to a widget area, but now you don’t change your themes or widgets to add explicit support for Widget Customizer.

No more Update button (usually)

Widget forms on the admin page have a familiar Save button. In previous versions of Widget Customizer, we adapted the Save button to be an Update button, since it wouldn’t actually save the widget but would just update the preview with the widget’s latest state. Having to click this Update button to see the preview update, however, was a poor user experience and was unnecessary. Therefore, we implemented a means of submitting the form to update the widget’s state in the preview whenever a field in the control is updated: the Update button was hidden and the spinner appears when a change is being synced to the preview.

The logic which does this live submission of the widget form as you interact with it, depends on the fields in the widget form to be the same fields which get returned when the widget form is sanitized. This is so that the fields can be aligned for being updated with their sanitized values. We couldn’t go the route of the widget forms on the widgets admin page, which actually get fully replaced with the newly-sanitized form, because this would interfere with the user quickly inputting into these fields.

So if a widget form dynamically adds or changes which input fields are included, the live-as-you-type-them updates will stop and the Update button will re-appear. When you click this button, the form will replace itself with the sanitized version just like on the widgets admin page. We also introduced new jQuery events to help handle these form changes…

New widget-added and widget-updated jQuery events

In the widgets admin page, when you add a new widget to a widget area, a widget template element gets cloned and then inserted into the selected widget area. This widget template is cloned in a way that any event handlers initially added to the widget template do not persist on any newly-added widgets. Any widgets previously-added to the widget areas would have these event handlers attached, as well as any dynamically-created fields (e.g. jQuery Chosen), but again, newly-added widgets fail to retain these. The same problem exists, however, whenever you update a widget: since the newly-sanitized widget form replaces the old one, any dynamic elements get lost. (This is why event delegation is often required.)

The same challenges here for the widgets admin page are also challenges for widgets in the customizer. And so in #19675 and #27491, we’ve introduced widget-added and widget-updated jQuery events which pass along the widget container as the second argument for the event handler. So to initialize and re-initialize a widget’s dynamic UI, you can now use these events in something like this:

jQuery( function ( $ ) {

	function init( widget_el, is_cloned ) {

		// Clean up from the clone which lost all events and data
		if ( is_cloned ) {
			widget_el.find( '.chosen-container' ).remove();
		}

		widget_el.find( 'select.dynamic-field-chosen-select' ).chosen( { width: '100%' } );
	}

	/**
	 * @param {jQuery.Event} e
	 * @param {jQuery} widget_el
	 */
	function on_form_update( e, widget_el ) {
		if ( 'dynamic_fields_test' === widget_el.find( 'input[name=&quot;id_base&quot;]' ).val() ) {
			init( widget_el, 'widget-added' === e.type );
		}
	}

	$( document ).on( 'widget-updated', on_form_update );
	$( document ).on( 'widget-added', on_form_update );

	$( '.widget:has(.dynamic-field-chosen-select)' ).each( function () {
		init( $( this ) );
	} );

} );

This is admittedly still not ideal, but it is much better than hacking the jQuery Ajax events to detect when a form update happens. In the future, I’m keen to see the widgets be revamped to use Backbone extensively.

Configuring widgets during a theme switch

We had a surprise late in the release (#27767) when it was reported that when you activate a new theme via the customizer, any changes made to the widget areas get lost when the theme is activated. This issue has been fixed, so now you can preview a theme and fully configure all of the widget areas before going live.

Beware of widgets and caching

We had another challenge in #27538 where widget changes previewed in the customizer would leak outside the preview and on the live site if the widget cached the output for performance. The core widgets Recent Posts and Recent Comments both cache their outputs in the object cache, and if a persistent object cache plugin is enabled (e.g. Memcached or APC) then the cache would get poisoned with the previewed widget, even though the widget’s changes weren’t saved. The same behavior would be experienced for widgets that use transients to cache the rendered output.

To help widgets do the right thing in regards to caching, we introduced a WP_Widget::is_preview() method which widgets should always check before they interact with the cache. For example, a widget’s widget()method which caches its output should do something like this:

if ( ! $this-&gt;is_preview() ) {
    $cached = wp_cache_get( 'foo-widget' );
    if ( ! empty( $cached ) ) {
        echo $cached;
        return;
    }
    ob_start();
}

extract( $args );
echo $before_widget;
// ...
echo $after_widget;

if ( ! $this-&gt;is_preview() ) {
    $cached = ob_get_flush();
    wp_cache_set( 'foo-widget', $cached );
}

Look forward to more improvements to widgets and the customizer in future releases!

#3-9, #appearance, #dev-notes, #widgets

Widget Customizer Feature-as-Plugin Merge Proposal

Widgets in WordPress provide an easy way to add functionality to predefined areas of your theme templates. However, once you add a widget to a sidebar you have to leave the WordPress admin to go back to the frontend to actually see how the updated widget appears in the sidebar on your site’s public frontend. While you are making these changes and experimenting with a widget, it could be completely broken and everyone visiting your site will see this broken widget since there is no core way to preview changes made to widgets. But WordPress also provides an excellent way to preview changes to various settings on your site via the (Theme) Customizer. Changes made when using the Customizer are not visible to site visitors until you hit Save & Publish. So what if widgets could be edited in the Customizer? That’s what the Widget Customizer plugin makes possible. No longer do you have to edit your widgets blind!

A widget control open the customizer

Each registered sidebar on your site gets its own section in the Customizer panel. Within each section, widgets appear in their sidebar order. The widget controls appear there just as they appear when editing widgets in the widgets admin page. Upon making a change to the widget control, pressing the form’s Update button causes the changes to appear in the preview window; no changes to the widgets are saved permanently for others to see until you hit the Customizer’s Save & Publish button. This goes for whether you’re adding a new widget, editing existing widgets, reordering widgets, dragging widgets to other sidebars, or even removing widgets from the sidebars entirely: all of these actions are previewable.

Adding a widget to a sidebar slides open a panel for browsing the available widgets, complete with the usual names and descriptions, but also featuring widget icons to help quickly identify widgets. The list of available widgets can also be filtered down with a search input.

When you remove a widget from a sidebar, it is not deleted. Instead, it is moved from an active sidebar to the “Inactive Widgets” sidebar which can currently be seen on the widgets admin page. As such, removing a widget now is the same as trashing a widget.

Adding a widget

Customizer control sections for sidebars are shown and hidden dynamically when the preview window is initially loaded or when navigating the site within the preview window, based on whether or not the sidebar gets rendered in the previewed URL. (You may not know this, but you can navigate your site by clicking links in the preview window.) Only sidebars which can be previewed will be shown in the customizer panel. Likewise, widgets that are not rendered in the preview (for example, by means of Jetpack’s Widget Visibility module) will appear in the Customizer as semi-transparent.

Accessibility has also been a key concern for Widget Customizer. The current keyboard navigation on the widgets admin page feels cumbersome, having to open screen options to enable a separate accessibility mode. We wanted to make re-ordering with widgets as seamless as possible. So now when any sidebar section is open, you can invoke a reorder mode to reveal up/down arrows to reorder widgets, as well as a subpanel to open for moving the widget to another sidebar entirely. (This feature is nearing completion in pull request #21.)

Here’s an older walkthrough of the plugin:

Live Previews

(This did not make it into WordPress 3.9 — that also means themes do not need to indicate support for the widgets customizer. Read more about Live Widget Previews: Widget Management in the Customizer in WordPress 3.9.)

While all themes and widgets should work with Widget Customizer, for the best experience the themes and widgets need to indicate they support live previews of sidebars and widgets. Without such support added, each change to a sidebar or widget will result in the preview window being refreshed, resulting in a delay before the changes can be seen (settings default to transport=refresh). To enable a much more responsive preview experience, themes and widgets should indicate that they support Widget Customizer live previews; this will update the relevant settings to use transport=postMessage, the updated widgets will be loaded via Ajax, and the widgets will be re-ordered via DOM manipulation.

All core widgets and themes distributed with WordPress core are supported by default. For other themes, simply include add_theme_support('widget-customizer') in your theme’s functions.php to opt-in. If your theme does some dynamic layout for a sidebar (like Twenty Thirteen uses jQuery Masonry), you’ll also need to then enqueue some JavaScript to listen for changes to the sidebar and reflow them when that happens; see the bundled support for Twenty Thirteen to see an example of what is required.

Along with themes needing to indicate support for live-previewable sidebars, widgets must also indicate that they support being live-previewed with Widget Customizer. When updating a widget, an Ajax call is made to re-render the widget with the latest changes, and then the widget element is replaced in the sidebar inside the preview. If a widget is purely static HTML with no associated script behaviors or dynamic stylesheets (like all widgets in core), then they can right-away indicate support for live previews simply by including 'customizer_support' => true in the array passed to WP_Widget::__construct(). As with sidebars, if a widget has dynamic behaviors which normally only get added when the page first loads (such as a widget which includes a carousel), then a script needs to be enqueued in the Customizer preview which will re-initialize the widget when a widget is changed.

The sidebar-updated and widget-updated events get triggered on wp.customize when sidebars and widgets get updated, each being passed the sidebar ID and the widget ID respectively as the first argument in the callbacks. For a full example demonstrating how to add theme support for live-previewing dynamic sidebars and how to add support for JS-initialized widgets, see this annotated Gist.

Core Tickets

A few Core tickets have been opened with patches to generally improve widgets and the customizer, and also to improve Widget Customizer itself:

  • #26633: Customizer form submission prevention impairs accessibility of links in customizer controls
  • #26061: Customizer settings with non-scalar values incorrectly trigger as changed
  • #26569: URLs exported to JavaScript in Customizer settings get double-encoded
  • #25368: Add temp hooks for Widgets UI Refresh plugin-as-feature
  • #26661: Add before/after hooks to override output of wp_widget_control()
  • #25419: Add support to widgets for icons and screenshots

User Tests

A couple user tests have been done, both of which went pretty well. More user testing is being queued up. Here’s the first user test video, though note it reflects an early rendition of the plugin:

[wpvideo tH9DTaey]

Remaining Issues

In addition to wrapping up the keyboard-accessible widget reordering (#21), there is the dilemma of how to handle wide widget form controls (#18); various solutions have been proposed for how to display a widget control which does not fit within the customizer panel.

The other open issues are enhancements or open questions which may or may not need actioning.

Contributors

While the plugin was first conceived by me (@westonruter) and I’ve been the lead developer, a ton of valuable input and contributions have come from @shaunandrews, @michael-arestad, from my X-Team colleagues (@johnregan3, @akeda, @topher1kenobe), and from others in the community (@bobbravo2, @topquarky, @ricardocorreia).

Development on the plugin has been done on GitHub, with the WordPress.org repo serving as a read-only mirror.

See Also

#customize, #feature-plugins, #merge, #proposal, #widgets

Better Widgets

The Widgets team has been busy. 🙂 Outside of the Widget Customizer plugin (posted about previously), we’re also working on some updates to the main wp-admin widgets screen through the Better Widgets plugin. This plugin does a bunch of things:

  • Available widgets have moved to the right side of the screen. The idea is that your widget areas (a.k.a. sidebars) should be the real focus of the screen — these are the things you can edit and manage. This may be a controversial change, as its the opposite of the menu screen (widgets closest cousin.)
  • Brings the widget icons from the Widget Customizer plugin to wp-admin.
  • Available widgets are now contained in a separately scrollable area. The goal is to help reduce to drag-and-scroll-and-scroll-and-scroll-and-drop problem that is so common from our initial research.
  • Widget descriptions are displayed in a single line, and truncated if they are too long. Clicking/Tapping on a widget expands the description (along with the area chooser from 3.8.)
  • Inactive Widgets are displayed below your active widget areas. This may be problematic as you have to drag-and-drop inactive widgets to active widget areas, but its an area we’d like to improve — maybe they should get an “area chooser”-like UI?
  • When editing a widget, the title is highlighted (using your current color scheme).
  • Clicking/Tapping on the “Save” button inside a widget now closes the widget and gives a quick confirmation message that the settings have been saved. This is based on some of our earlier user tests.
  • You’ve always been able to drag an active/inactive widget over the list of available widgets to deactivate it (yes, really!), but its been ugly. We’ve made it a little more tolerable. Give it a try.

The plugin is still very young, but we’re looking to the community to get some interested from designers, developers, and testers. Please, install the plugin and play around. If you’d like to help us improve widgets, please join us every Monday at 20:00 UTC in #wordpress-ui — you can also drop your Skype nick below and we’ll add you to our ongoing chat.

Some things to keep in mind when testing the Better Widgets plugin:

  • Responsive styles are essentially broken. Its on our short list, but we haven’t gotten to it yet.
  • The code is quick and dirrty — I’ve been the only developer committing code. Please, lets change this!
  • Some of this may look familiar to early MP6 adopters — this code comes from an earlier version of MP6, and was removed before the MP6 merge into 3.8.
  • We need accessibility help! Keyboard navigation is a must. I’d love to ditch the separate “accessibility mode” altogether and make it accessible out of the box.

#widgets

Widgets Area Chooser – 3.8 Proposal

Placing widgets with drag-and-drop can be tedious and annoying — especially if you have lots of sidebars on which to drop widgets. The Widgets team has been working on a few solutions (for this problem, and more), including redesigning the wp-admin widgets interface and adding the ability to manage widgets from within the customizer. These projects are still ongoing, and not ready for 3.8. However, along the way we’ve found a few incremental changes which improve the overall experience of working with widgets. Some of these improvements have made their way into MP6. Others involve more functional changes which don’t belong in MP6. This plugin is one of those improvements.

The Widgets Area Chooser is available at: https://wordpress.org/plugins/widget-area-chooser/

The Problem
Dragging widgets from the available widgets in the top-left, to a sidebar “below the fold” is hard. Almost impossible. Dragging widgets on a touch screen device is also difficult.

The Solution
Clicking (or tapping) on an available widget brings up a list of available sidebars that you can place the widget in to — its pretty simple, and works great on touch devices.

Accessibility
There’s also the accessibility problems that drag-and-drop introduces, which necessitates the need for the separate (and often neglected) Accessibility Mode. This plugin provides a much easier way for those with screen readers to add new widgets without having to drag-and-drop. In fact, this could be the first step towards removing the need for an Accessibility Mode for widgets.

Demonstration
Here’s what the chooser looks like:

Here’s a quick video of the plugin in action:

[wpvideo hHcvYfGT]

Please let us know what you think!

#3-8, #widgets