Performance improvements for images in WordPress 4.5

WordPress 4.5 includes a few performance enhancements for images.

Increased image compression for custom sizes

WordPress 4.5 increases the amount of compression applied to intermediate sizes by changing the default quality in WP_Image_Editor from 90 to 82. As noted in the proposal for this change, this results in a noticeable reduction in file sizes with little change in visual quality. Developers can override the default image quality value using the wp_editor_set_quality filter.

Improved resizing settings for ImageMagick

For sites making use of ImageMagick, we’ve reduced file sizes further by resizing images  more efficiently in WP_Image_Editor_Imagick and by stripping extraneous metadata using the new WP_Image_Editor_Imagick::strip_image() method.

For now, ‘icc’ and ‘icm’ color profiles are retained, along with ‘exif’, ‘xmp’, and ‘iptc’ profiles, which can contain copyright and orientation data. Those who want to retain additional metadata can disable profile stripping by adding a callback function to the image_strip_meta hook that returns false.

Note that the original full sized images uploaded to WordPress are unaffected by these changes.

Introduction of wp_get_upload_dir()

As Jeremy Felt mentioned in his post on Multisite changes, wp_upload_dir() received a major performance overhaul in this release. Those changes were pared with the addition of a new function, wp_get_upload_dir(), which can be used as a more performant way to display information about the uploads directory on the front end. This is particularly useful when building URLs for images in templates. (See #34359)

#4-5, #dev-notes, #images, #media, #optimization, #performance

To the Directors of White Space

Dear Sirs,

It has come to my attention that our White Space is running wild. Would it be possible to tame that beast back into submission please.

Seriously, I would like to propose adding another rule to the coding standards: when outputting HTML directly, leave all white space in PHP. Currently we do something like this:

if ( something ) {
    <div id=...>some more</div>
    <div id=...>even more</div>

That’s all nicely readable on the PHP side but the outputted HTML is surrounded by quite a bit of redundant space. What I’m proposing is to stop/start PHP immediately around the HTML:

if ( something ) {
    ?><div id=...>some more</div><?php
    ?><div id=...>even more</div><?php

This doesn’t impact the readability on the PHP side and produces “tight” HTML with no white spaces as it should be for production. In fact the HTML source would be “pseudo-minified”.

Of course the readability of the HTML source will be affected but not that much. Currently our HTML output is all over the place which makes it pretty hard to read. Considering that most people never look at the HTML source directly any more thanks to FIrebug and friends and all coding oriented text editors have a function to reformat HTML, I believe outputting “minified” HTML is an advantage. This also reduces the size of our output from 3% to 15% depending on the page.

Of course I’m not proposing for everybody to rush and reformat the whole codebase if this is accepted. But we can apply it to new code and clean up functions that are being patched for other reasons.


Idea: We should do a branch prior to rel…

Idea: We should do a branch prior to release (we do this anyway) and then commit to that branch a minimization of all JS and CSS, so trunk always has pretty-formatted, and release branches always have minimized.

Or this might not be needed with Andrew’s new stuff.


Script loader updates

There are several updates to the script loader currently in WordPress 2.8-bleeding-edge that enhance and optimize loading of external JavaScript and CSS files.

Probably the most important change is that scripts can be queued for loading in the footer for both the admin and the front-end. This is done with an optional argument. To enqueue a script for the footer:

wp_enqueue_script( 'name', 'url/to/file.js', array('dependency'), 'version', true );

where “true” means enqueue for the footer (“false” is the default and is optional).

When a script is enqueued for the footer all dependencies will be added (if not already present) and will be printed before the script. Some may be in the head, others also in the footer. By default only jQuery is printed in the head but when a script is enqueued for the head, all dependencies would also be printed in the head. Almost all external scripts would run onload or after the page has loaded, so there’s no real need to queue anything for the head.

Scripts queued for the front-end footer depend on wp_footer(); being present in the current theme. Unfortunately some themes don’t include it. The best way to remedy this would be to bring awareness among users and theme designers as suggested by several plugin developers.

To make queueing of scripts easier two new actions have been added: "wp_enqueue_scripts" that runs in the front-end head where all is_page(), is_home(), etc. functions are available and "admin_enqueue_scripts" that runs in the admin head and has the current page hook as argument, so scripts can be queued only for specific pages.

Another major new feature is that all core admin scripts are concatenated and compressed before sending them to the browser. This feature can easily be extended to include scripts added by plugins and to use server side caching, however that would require some changes to the server settings (.htaccess on Apache).

Since compression from php can be problematic on some hosts there are several “switches” (constants) that manage it: define('CONCATENATE_SCRIPTS', false); would turn off both concatenating and compressing of all scripts. It’s intended for script debugging, define('COMPRESS_SCRIPTS', false); can be used to turn off compression for JavaScript and define('COMPRESS_CSS', false); for CSS files. Compression is set to “deflate” by default since it’s faster and uses a little less server resources. Gzip can be forced by setting define('ENFORCE_GZIP', true);

There is a test if compressing from php works as expected on the server and whether the server compresses scripts by default. It runs only once and saves the result in an option “can_compress_scripts”. It would run again if the option is deleted.

In addition all core scripts are minified. All custom scripts are included in two versions: .dev.js is the non-minified script and .js is the minified one. The constant define('SCRIPT_DEBUG', true); would load the .dev.js versions of the scripts making them easier to debug.

Possible changes: removing the COMPRESS_CSS switch and using only COMPRESS_SCRIPTS, using deflate for compression but adding the gzip file header and serving it as “Content-Encoding gzip” since it seems more compatible with the various web servers and proxyes (all modern browsers support deflate well).


Optimizing script loading, implementation

The “first run” of the script loading optimization is in trunk. It uses both methods: splits the scripts queue in head and footer parts, then concatenates and compresses them before sending them to the browser. Most CSS is also concatenated and compressed.

There are two new constants that disable concatenating and compression: CONCATENATE_SCRIPTS and COMPRESS_SCRIPTS. Setting the first to false would disable both concatenating and compression, the second disables compression only. There is also a simple AJAX method to test if compressing from PHP works as expected on the server. It is run only once and the result is saved as an option. It will run again if that option is deleted.

For plugin authors there are two new actions in the admin footer: do_action('admin_print_footer_scripts'); and do_action("admin_footer-$hook_suffix");. The order of execution is:

  1. "admin_footer" can be used to print scripts before the default footer scripts
  2. "admin_print_footer_scripts" used to print the default scripts followed by any external scripts that were queued for the footer
  3. "admin_footer-$hook_suffix" can be used to print scripts that should appear on a specific admin page only

The preferable way for plugins to add scripts to admin pages would be either to enqueue them properly (which is a must if the script depends on a default script) or to use the $hook_suffix hooks to add them only where needed.

New hook(s) may also be needed in the functions dealing with splitting or concatenating the script queue. Suggestions are welcome either here as comments or on trac as enhancements tickets.

Update: added do_action('admin_enqueue_scripts', $hook_suffix); allowing plugins to easily queue scripts on the exact pages. To find out the value for $hook_suffix on a specific page, echo it from your function and visit the page.

Queueing a script for the footer follows the same syntax as in script loader:

$scripts->add( 'handle', 'full/url/to/file.js', array( 'dependency' ), 'version' );
$scripts->add_data( 'handle', 'group', 1 );

Group 1 means queue for the footer, group 0 (zero) is the default.


Optimizing script loading, part II

After more tests and research it seems the best two options for the WordPress admin are either minifying all scripts and loading most in the footer or concatenating and compressing them on per page basis.

Minifying and loading scripts in the footer gives slightly slower performance with both cold and primed cache and would depend on the ability to set appropriate caching headers on the server. On the plus side this method would be compatible with the current plugins and wouldn’t introduce higher server load.

On the down side the loading speed improvement with primed cache would depend on the availability of mod_headers and/or mod_expires (presuming most installations are on Apache). It seems many hosts have either one or both modules installed but that still leaves a lot of installations without proper caching headers. In these cases the browser would keep checking for updated content which would increase the loading time with primed cache considerably.

Concatenating and compressing all scripts would give better speed improvement and we will be able to set all needed headers. The last couple of days I’ve been testing a method that uses a separate php file similar to how the Gears manifest is produced. This is the same basic method used by many “website php compressors”, a stand-alone php function that gets as argument the names of the needed scripts then concatenates and compresses them. It doesn’t use server side caching (that proved to be problematic on some servers) since the cold cache page hits on the admin are relatively few.

The advantage is that WordPress is not run second time on every cold cache page load so it uses a lot less server CPU time and memory. The disadvantage is that it would work only for the default scripts whose paths are included in the script loader. This also seems to be compatible with all existing plugins as any additional scripts are loaded after the single script and all requirements are satisfied.

Another disadvantage is that a few hosts seem to compress all php output in a non-standard way that may result in double compression. This method would also need “Optimization options” screen with a few checkboxes that would allow the user to enable/disable the concatenating and compression as it won’t be needed when using Gears.


Optimizing script loading

WordPress is using more and more JavaScript. A lot of features use AJAX and nearly all UI customizations and enhancements depend on it. In 2.7 there is some optimization for caching, especially when Turbo (Gears) is used all pages load in under 1 sec. However the cache control was left to the server and there seem to be a lot of servers with less than perfect settings for it.

There are many ways to optimize script loading in 2.8. Listing several of them roughly in order of ease of implementation/most effective:

  • Minify (but not “pack”) all js files. Although the file size is larger, minified js loads faster than “packed” js, as it takes the browser some time to “unpack” it. Here’s a nice post on the subject. In 2.8 we should replace all “packed” js with minified and also minify all custom js when building the release.
  • Load most js in the footer. Even with 15 – 20 js files a page will load a lot faster when most are in the footer. In theory we can load all but jQuery in the footer. There’s a ticket and patch for the script loader to do this. However in practice that will interfere with many plugins that load any of the default js files on admin pages. Seems that if we load jQuery and any jQuery UI parts in the head, the great majority of plugins won’t be affected. Another option would be to queue all UI parts for the footer but move them to the head if a plugin loaded js lists them as dependencies (that would complicate the script loader a bit).
  • Setting custom caching headers in .htaccess. Although this may not work on all servers, chances are that the great majority of installations would benefit. There are several settings that can be added: the “expire far in the future” is a must, either configuring or disabling Etags for the js, css and image files is advisable, also several of the HTTP 1.0 cache control headers would be beneficial. Of course this would need testing on different production servers, but it would degrade gracefully (just wouldn’t work), so counting it as “must implement”.
  • Adding all js and css files in both compressed and uncompressed state and setting the server (.htaccess) to use the compressed versions whenever possible. This would greatly improve the initial loading of all scripts (cold cache), but is not as trivial to set up as the cache control headers. Still worth a try even if not all WordPress installations would benefit. It is possible to set the server to compress “on the fly” or compress by using php, but both methods use a lot more of server CPU time/memory on each page load. Since this affects only cold cache hits, not sure if it’s worth it. The typical user case for the admin is the same user on the same computer and in the same browser accessing the admin at least several times per week. So making sure caching works best is a priority.
  • Concatenating all default js files on per page basis so there’s only one js file to load. This combined with compressing the file would give the fastest loading time for both cold and hot cache, although for hot cache with proper cache control the difference is almost unnoticeable. It is possible to do this when building the release and include two versions of each file: compressed and uncompressed, then use .htaccess to output the compressed version when possible. However when there is a plugin that loads js and depends on other default js files, the performance degrades quickly and most blogs seem to have at least one such plugin. The workaround is to dynamically generate the concatenated file for each page, compress it and store it on the server. That would mean doing something similar to WP Super Cache but only for the js and css files in the admin. There seem to be at least one plugin that attempts to do this, also several regular WordPress contributors are interested in making such plugins. Perhaps for now it’s better to leave this for a plugin as there seem to be many server configurations where it would be difficult to set up or won’t work properly. We can also add some hooks in script loader to facilitate this if needed.

In terms of performance the concatenating and compressing of all js and css files for each page gives the best results, closely followed by the loading of most js files in the footer. If we decide not to do the concatenating/compressing, we should revisit it in about a year (for WordPress 3.0?) as most Web hosts would have upgraded to php 5 by then. The actual php version wouldn’t affect this, but it seems that while upgrading the hosts also improve their server and php configurations too.