Fast previewing changes to Menus in the Customizer

The foundation of the Customizer is built on the concept of the “setting”, an abstract representation of anything in WordPress: options, theme mods, posts, terms, and so on. Settings are linked to controls which are responsible for manipulating them. When a setting is changed it gets pushed into the preview window so that the modification can be reviewed before being published live.

There are two ways (transports) for how a setting can be pushed into the preview: refresh and postMessage. The refresh transport does a full page refresh with the modified settings applied, and so it relies on PHP to apply the setting changes. The refresh transport is designed to be compatible with all themes, but for heavy sites refresh can be excruciating slow and taxing on the server. For this reason, the postMessage transport was introduced for previewing changes. This transport uses JavaScript to apply the changes instantly in the preview without any Ajax requests or PHP calls. This is great, but themes have to opt-in to postMessage transport for settings because they must also include JavaScript code which duplicates the logic which is already being executed on the server by PHP. The postMessage transport violates the DRY principle.

With that background out of the way…

Partial Refresh of Menus

In WordPress 4.3 menu management is being added to the Customizer, extending the types of objects that the Customizer can manipulate to include nav_menu taxonomy terms and nav_menu_item posts. We wanted this menu management experience to be fast so it had to avoid the slow full-page refresh transport. Nevertheless, the postMessage transport wasn’t possible to use either since we wouldn’t be able to duplicate all of the PHP menu walker logic and theme/plugin extensions for rendering menus purely in JavaScript. We also wanted to enable fast previewing of menu changes by default. So we implemented a postMessage/refresh hybrid approach which uses postMessage to sync the menus settings to the preview, and then the preview does an Ajax request to render just the contents of the menu container, and this Ajax response is then inserted into the DOM to replace the previous menu container. The technical name for this approach we have been calling “partial refresh”, but you can call it “fast preview”.

When the Customizer determines it can’t do a partial refresh, it falls back to performing a full page refresh. Conditions for why a menu change will get a full page refresh include:

  • if the menu was just assigned to a location for the first time or removed from being assigned to a location, since the theme may have some changes to the layout that it may need to perform;
  • or if wp_nav_menu() is called with with echo being false, since the theme may be doing some string replacement in this case which we can’t replicate in the partial-refresh Ajax request;
  • or if wp_nav_menu() is called with with a non-JSON serializable fallback_cb, such as a closure or a object method—this arg has to be empty or a string;
  • or if wp_nav_menu() is called with with a walker object instance, as opposed to a class name string;
  • or if wp_nav_menu() is called without a theme_location arg and the menu arg is not an term ID or a menu object;
  • or if the menu selected in a Custom Menu widget is changed, since partial refresh for widgets is not yet supported (however, see feature plugin)—subsequent changes to the menu will result in fast partial refreshes if the above conditions

When a menu is being updated via partial refresh, the menu container will receive a classname customize-partial-refreshing, which by default changes the opacity to 25%.

Theme Support

The fast preview for menu changes is being enabled by default so that theme authors don’t have to opt-in to the functionality like they do for instant preview (postMessage transport). In general this should be fine. However, if the theme includes some JavaScript which dynamically manipulates the rendered menu container, such as adding event handlers for expanding/collapsing submenus, then the theme would need to include some JS to re-apply the modifications once the menu is partial-refreshed, as the old DOM for the menu container gets replaced.

The Twenty Fifteen theme actually uses JS to expand/collapse submenu items, so it needed a patch in 4.3 to re-setup a partial-refreshed main nav menu. In addition to firing the menu container DOM setup logic once at jQuery.ready(), it now invokes the functionality whenever the menu is partial-refreshed. See the initMainNavigation() JS function.

So in general, if your theme needs to dynamically initialize menus with JavaScript, the pattern for the initialization code should be:

jQuery(function($) {
	function initMainNavigation( container ) {
		/* set up container... */
	}
	initMainNavigation( $( '.main-navigation' ) );

	$( document ).on( 'customize-preview-menu-refreshed', function( e, params ) {
		if ( 'primary' === params.wpNavMenuArgs.theme_location ) {
			initMainNavigation( params.newContainer );
			/* optionally sync a previous menu state from params.oldContainer... */
		}
	});
});

The params being passed to the event handler consists of the following properties:

  • newContainer: jQuery object containing the new menu container element retrieved from Ajax; this is what you would manipulate to initialize.
  • oldContainer: the previous jQuery object holding the element for the replaced menu container; this is useful if there is any state in the old menu that should persist in the new menu, such as which submenus are expanded (as in Twenty Fifteen).
  • wpNavMenuArgs: The array of arguments passed to wp_nav_menu() in the template, such as template_location.
  • instanceNumber: The index for which wp_nav_menu() call being updated.

The Future

Partial refresh is an “experimental transport” introduced here specifically for menus. In 4.4 we hope to generalize the framework so that any setting can be setup for fast preview, starting with widgets (see #27355). There is a Customize Partial Refresh feature plugin that implements widget partial refresh in a beta state. This partial refresh functionality I’ve also identified as being critical for Customizer Transactions.

#4-3, #customize, #dev-notes, #feature-plugins, #feature-selective-refresh, #kickoff, #menu-customizer, #menus, #partial-refresh, #proposal

Proposal: Customizer Transactions

Note: Note: This post has been superseded by Customize Changesets (formerly Transactions) Merge Proposal. See also Customize Changesets Technical Design Decisions.

#customize, #feature-selective-refresh, #javascript, #json-api, #partial-refresh, #rest-api

You may be familiar with transactions in a database context. The idea is simple: once you start a transaction, any change made to the database in the current session will not be applied to other database sessions until a transaction COMMIT is done. Changes performed when the transaction is open will be reflected in any SQL SELECT queries made, and if you decide that you do not want to persist the changes in the transaction in the database, you can simply do ROLLBACK and the changes will be discarded. And actually WordPress already uses MySQL transactions, but only in unit tests: a rollback is done after each test is torn down to restore the database state for the next test.

The parallels between database transactions and the WordPress Customizer are clear. When you open the Customizer you are starting a “settings transaction”. Any changes made to the settings in the Customizer get reflected in the preview only, and they get written (committed) to the database only when the user hits “Save & Publish”.

As good as the Customizer currently is, the way it has been implemented means that there are limitations on what we can do with it.

Current Limitations

The existence of modified settings in the Customizer is restricted to the life of a browser window. When a user changes a control in the Customizer and a setting is modified (with transport=refresh), an Ajax request is made with the changed settings data POSTed to the previewed URL. The Customizer then boots up and adds the setting preview filters based on what it sees in $_POST['customized'] so that the changes are reflected when WordPress builds the page. When this Ajax response is received, the Customizer JS code then writes the response to the iframe via document.write().

There are a few downsides to this current approach:

One problem is that if the user navigates away from the Customizer, they lose their drafted settings. To get around this, an AYS dialog was added in #25439, but this still doesn’t account for browser crashes or system failures. It would be ideal if the settings could persist in the same way as when drafting a post.

Another downside is that whenever the preview needs to refresh it has to re-send all the modified settings so that the Customizer preview will have them available to add to the filters, since the Customized settings data is not persisted in WordPress in any way. There’s a performance hit to continually send all data with each request, which was partially improved with #28580.

Additional problems stem from the Ajax POST + document.write() approach to refreshing the preview. Since the Customizer iframe starts out at about:blank and the HTML is written to from the document at customize.php, much of the context for the document in the iframe gets inherited from the parent window. This means that window.location in the preview window is the same as in the parent window: /wp-admin/customize.php. Needless to say, this means that JavaScript code running in the Preview will not run as expected (e.g. #23225).

The Customizer preview intercepts all click events and sends the intended URL to the parent window so that the Customizer can initiate the request to refresh the preview. The only way to change the current page in the preview is by clicking a standard links with a URL; any form submissions in the preview are completely disabled, resulting in the search results page not being reachable from within the preview (#20714). Any navigation system that uses JavaScript to change the window’s location also will fail, for instance using a dropdown.

The current unique method for refreshing the preview worked fine when the Customizer was limited to a handful of settings. But now as more and more of WordPress is being added to the Customizer, and now that themes are increasingly leveraging JavaScript, we need a more robust approach to implementing the Customizer which will solve the above challenges and provide new opportunities.

Customizer Transactions

The proposal is that we introduce persisted Customizer settings, in other words “Customizer transactions”. Here’s how it may work:

When opening the Customizer for the first time, a transaction UUID is generated. Whenever a setting changes, an Ajax request sends the updated setting to WordPress to be persisted in a wp_transaction post which has a post_name corresponding to that UUID (or such a transaction post is created on the fly if not existing already). Any changes made in the Customizer then get amended to the same wp_transaction post, which has a key/value JSON blob as its post_content.

When a user hits the Save & Publish button, the underlying wp_transaction post gets a post status change to publish. When transitioning into this status, each of the settings in the transaction at that point get saved to the database—they get committed.

Instead of using an Ajax POST to send the customized settings to the preview, we then only have to reference the transaction UUID when loading URLs into the Customizer preview. What this means is that we no longer have to use a blank iframe but can load the window with the natural URL for what is being previewed (#30028), but just with the transaction UUID query parameter tacked on.

When this transaction UUID query parameter is present, filters get added to amend all URLs generated in the preview to also include this UUID, so the transaction context is persisted as the user navigates around the site in the preview window. Forms also get this transaction UUID added as another input element, so any form submissions will also keep the preview inside the transaction. Additionally, WP Ajax requests are intercepted to tack on the transaction UUID so that now even Ajax requests can be previewed in the Customizer without any extra work.

Now that the document in the preview window is actually at the URL being previewed (as opposed to about:blank), refreshing the preview is greatly simplified: instead of capturing scroll position, doing Ajax POST, writing the response with document.write(), and restoring the scroll position—now the preview window just has to do a simple location.reload(). JavaScript now runs in the expected context, and full JS applications can be previewed in the Customizer.

As noted above, each time the Customizer is opened and a setting is updated the first time, a wp_transaction post is created with a draft status, and this post gets updated each time a setting is changed during that Customizer session. You also can open the Customizer as a whole (at customize.php) with this transaction UUID supplied and that settings in that existing transaction draft will be loaded. This means you can draft Customizer settings and return to them later, or make some changes and then send it along to another user to finalize (realtime collaboration would be possible as well with some heartbeat integration, or else a locking mechanism would make sense). The capability to publish wp_transaction posts could be restricted to an administrator role, with other roles being able to save posts with a pending status to submit for review.

Also as noted above, the point at which the settings in a transaction get saved (committed) to the database is when the wp_transaction post transitions to a publish status. This being the case it naturally allows for transaction posts to be scheduled to apply in the future. If you want to make a bunch of changes to your site appear at midnight on Saturday, you could go in on Friday and add/remove widgets, change background images, and do anything else the Customizer allows and then have this transaction be scheduled for the desired time. When it publishes, all of the settings would go live on the site. This resolves #28721.

With each Customizer session resulting in a new transaction post being created, then there is automatically a Customizer revision history (see #31089). Every transaction that has a publish post status is a change that went live on the site.

Another side benefit to reworking the Customizer preview to load via a natural URL with the transaction UUID supplied is that there aren’t any intrinsic capabilities needed to preview a transaction on the site. A setting change gets authorized at the time of the change, and the sanitized setting is then persisted in the transaction post. The preview then just applies the pre-authorized and pre-sanitized settings. The interesting side-effect of this is that it means Customizer previews (frontend URLs with the transaction UUID amended) can be shared with anonymous users to review. You can pop open the URL in the preview iframe into a new window and share it with any user for review, and they don’t need the capability to customize.

Lastly, something else that motivated my investigation into Customizer transactions is thinking about how the Customizer will relate to the upcoming REST API. How can the REST API be improved with the Customizer? Where do they intersect? If the REST API provides a transactions endpoint for doing CRUD operations on Customizer settings, and if the REST API also has global recognition for a customize_transaction_uuid query parameter in all requests, then it becomes possible for the Customizer to be used to preview changes in applications that merely interact with the JSON REST API, as long as they include the transaction UUID in the requests.

Partial Refresh

There’s one drawback I’ve encountered when implementing a patch for what I’ve described above. As noted above, when a setting has a refresh transport, the preview window now does a regular location.reload(). When this happens, there is a momentary “flash of unloaded content” (white screen) which currently doesn’t happen when document.write() is employed to refresh the preview window. I’m not sure why this is, other than maybe document.write() initiates a synchronous DOM operation, whereas doing location.reload() initiates an asynchronous one. I’ve tried doing output buffering as well, to try to make sure the response gets sent all at once. But I haven’t had success. This is the current refresh behavior:

If no solution can be found for the white-screen-flash-during-reload issue, there is an alternative (besides the postMessage transport) which would provide an even better experience than even now with the “seamless” full page refresh: partial refresh (#27355).

When a setting change can’t be previewed purely with JavaScript (via postMessage), or it doesn’t make sense to re-implement all of the PHP logic in JS (which is not DRY), the Customizer currently necessitates a full refresh of the entire page. With the proposed partial refresh transport, however, only the container element(s) in which the setting appears in the preview would get fetched from the server via Ajax and inserted into the DOM. This is much faster than having to refresh the entire page, and it retains the overall document state (e.g. whether the sidebar is expanded or not).

There are challenges for implementing partial refresh in a way that it can be enabled by default, however. When implementing partial refresh support for widgets in the Widget Customizer feature-as-plugin for 3.9, I found that themes had to explicitly opt-in to partial-refreshed widgets because a widget could be inside a sidebar that has a dynamic layout (e.g. jQuery Masonry) or the widget may have JS-driven functionality that has to be re-initialized when updated partial is injected. So partial refresh for widgets was removed from being included in WordPress 3.9, but the functionality has recently been resurrected in the Customize Partial Refresh plugin. More research is needed into how much partial refresh we can have enabled by default, and where we need explicit opt-in.

Call for Feedback

So there’s a lot of exciting possibilities introduced with Customizer transactions. I’d love to hear what you think. I have an working patch written and it exists in a pull request on GitHub. I welcome comments there on the PR. Naturally, the changes would need to be split up into smaller patches for committing to SVN.

Related tickets:

  • #30937: Add Customizer transactions (main ticket)
  • #30028: Load Customizer preview iframe with natural URL
  • #30936: Dynamically create WP_Customize_Settings for settings created on JS client
  • #27355: Customizer: Add framework for partial preview refreshes
  • #20714: Theme customizer: Impossible to preview a search results page
  • #23225: Customizer is Incompatible with jQuery UI Tabs.
  • #28721: Scheduled changes for the customizer
  • #31089: Customizer revisions
  • #31517: Customizer: show a notice after attempting to navigate to external links in live previews

Appendix: Why not just use MySQL transactions?

Something interesting to investigate for the future would if we could take this another (lower) level and actually use MySQL transactions for the Customizer. This would make the Customizer much easier to extend beyond options and theme mods, as the Customizer could just start a MySQL transaction and when a setting is changed, just keep a log of any INSERT/UPDATE/DELETE statement performed during a MySQL transaction. They can then be re-played whenever the preview reloads, and then followed by a COMMIT when the Customizer is saved. These SQL statements can be saved in the wp_transaction post, as opposed to the JSON blob containing the key/value settings data. Or the use of MySQL transactions could go deeper and the SAVEPOINT could be utilized to store the transaction natively in MySQL.

But there are some concerns about using MySQL transactions: first, there’s the question of compatibility and whether MySQL transactions would be available on the wide array of hosting environments where WordPress runs, and what MySQL storage engines are used. Then there’s the question of how conflicts would be resolved when the auto-incremented IDs in the transaction diverge from those outside. And also there’s the concern of storing SQL statements the wp_transaction post’s post_content, and how this might introduce vulnerabilities. Lastly, if we use MySQL transactions as opposed to storing the staged settings in a wp_transaction post, then we’d miss out on being able to inspect the contents of a transaction to see what changes are contained within it.

In any case, using MySQL transactions would be an interesting alternative to the current WP_Customize_Setting abstraction.

#customize, #feature-selective-refresh, #javascript, #json-api, #partial-refresh, #rest-api