Adam Silverstein 4:33 pm on December 9, 2022
Tags: core ( 598 ), feature projects ( 51 ), javascript ( 128 ), performance ( 346 ), proposal ( 37 )

Enhancing the Scripts API with a loading strategy

Overview

This post outlines a proposal to add a script loading strategy enhancementenhancement Enhancements are simple improvements to WordPress, such as the addition of a hook, a new feature, or an improvement to an existing feature. to coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.’s existing Scripts APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways..

The underlying goal of this effort is to make it easier for WordPress pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and theme developers and core to use “modern” loading approaches for JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. (like defer and async), which will help WordPress sites load faster and operate more smoothly, giving users a better experience.

Why add a loading strategy?

Data from the Web Almanac project (query) indicates that render blocking JavaScript is a significant problem on the web, with 77% of mobile pages having render-blocking scripts in the document <head> . This query shows that approximately 40% of WordPress sites stand to benefit from deferring additional scripts. Adding defer or async to script tags enables script loading without “blocking” the rest of the page load, resulting in more responsive sites overall better user experience.

Currently WordPress core as well as plugins and themes register scripts with the wp_enqueue_script and/or wp_register_script functions. Although these functions include the ability to control the placement of the script (with the in_footer parameter), they don’t include support for adding modern attributes such as defer or async to script tags.

To add async or defer today, developers must resort to less flexible and more fragile approaches, such as directly filtering the tags at the point of output (using the script_loader_tag filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.), or handling the tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) output directly using wp_print_script_tag and the wp_script_attributes filter.

Using the first approach and directly filtering the tag can easily break: for example if two plugins both try to filter a tag, or if a tag has unexpected attributes already (eg. adding defer to a tag that already has async). Using the the second approach developers must carefully handle dependencies and output manually – things that that the Scripts API usually helps take care of.

How the loading strategy works

Developers specify a loading strategy when registering or enqueueing a script. For example, a defer strategy can be specified when the script isn’t required immediately during the page load cycle. WordPress will then determine which scripts can actually use a strategy based on logic for each strategy. For example, to ensure that scripts are executed in the order they are enqueued, defer can only be used on a script if every script that depends on that script can also be deferred. Inline script tags added with wp_add_inline_script would also be considered to ensure proper execution order.

The implementation would come with several initial built-in loading strategies: defer, async, and the default blocking behavior.

Out of scope for this feature

The loading strategy does not enable direct control of script tag attributes. This idea was originally proposed 10 years ago in #22249 and several approaches were considered on that ticketticket Created for both bug reports and feature development on the bug tracker. including a script attribute filter. This proposal takes a step back and aims to solve the script loading strategy aspect more comprehensively and directly while avoiding exposing the potential complications of direct attribute control.

It is worth noting that it is already possible to control attributes on wp_enqueue_script tags directly using the script_loader_tag filter. However, this is a bit of a “brute force” approach which is limited and fragile because it doesn’t consider dependencies and multiple plugins can take conflicting actions on the same tag.

What are potential concerns with this feature?

One big concern with adding this feature to the WordPress Script API is potentially introducing a breaking change. wp_enqueue_script is a fundamental API in WordPress core, and any breaking changes could have widespread implications. Possible breakage is a possible reason that adding custom attributes as proposed in #22249 was never added to core.

This new proposal aims to ensure that there is 100% backwards compatibility, resulting in zero risk of breakage. The loading strategy will ensure that all existing uses continue to function as expected; for example, passing the boolean in_footer attribute will still control script placement. In addition, it will ensure that scripts continue to be executed in the order they are enqueued – as described above in the “How the loading strategy works” section.

Conclusion and Next Steps

Giving developers the ability to specify a loading strategy will enable them to use more advanced JavaScript loading methods while still ensuring that enqueued scripts are executed in the correct order. A “strategy” approach is also forward thinking: as the web evolves, new strategies can be developed and made available to WordPress developers. After gathering feedback, we will proceed to discussing the implementation approach and, ultimately, proposing a patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing..

Have you tried using defer or async with WordPress (or do you already)? How do you think this enhancement would change that? Please leave your feedback about this proposal in the comments below and if you can, join us at our weekly performance team chats, where we are likely to discuss this proposal in the future.

Thanks to @flixos90, @tweetythierry and @mxbclang for help writing and reviewing this post and for the many contributors who have added to the discussion around this enhancement already.

#core, #feature-projects, #javascript, #performance, #proposal

pingram 5:40 pm on December 9, 2022

Ooh yes, gets my thumbs up!
Mat Lipe 6:11 pm on December 9, 2022

WP script handling is assumed to automatically load scripts in the correct order based on dependencies. One of the problems with using the existing filters to add a `defer` tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) to a script is the dependency hierarchy execution order. The filtered script gets moved in execution order and can have side effects. The biggest value I see adding the loading strategy, is we can automatically adjust lower script dependencies to fix the load order when using `defer`. `async` will still have an unpredictable order, but that is the nature of asynchronous calls.

Removing filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. collision is also a nice bonus with using a loading strategy.

Assuming the existing filters will remain intact (obviously) and new or existing filters will allow adjusting the values sent as “strategy”, I don’t see a big lift with updating 3rd party wrappers to move away from the old filters to the system.

Lets do it. 🙂
Joseph Scott 10:13 pm on December 9, 2022

In general I support the concept.

Adding defer or async to script tags enables script loading without “blocking” the rest of the page load, resulting in more responsive sites overall better user experience.

This is true, but it leaves out what can happen with script execution. I recommend the diagram at https://html.spec.whatwg.org/#the-script-element for a good visual representation of the differences.

In addition to the ordering issue you brought up there an issue with when the script can be executed. With defer both the script download and execution avoid blocking the HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. parser. With async only the download portion avoids blocking, and execution could happen at any point, even before the HTML parser has completed.

Because of this difference I generally recommend that people not use the async attribute at all. “Prefer defer” is a handy way to remember this 🙂

I bring this up because I want to avoid confusing people who might think that async makes script execution avoid blocking the HTML parser. To ensure that behavior you need to use defer instead.
- Adam Silverstein 11:26 pm on December 9, 2022
  
  … Because of this difference I generally recommend that people not use the async attribute at all. “Prefer defer” is a handy way to remember this
  
  Thanks for the feedback. I agree, `defer` is generally preferable! Still, I have seen some legitimate use cases for `async`, especially for standalone scripts that want to execute as quickly as possible while still minimizing impact on the page load. For this reason it makes sense to support `async` out of the box while still encouraging developers “prefer defer”.
  - Sérgio Gomes 10:44 am on December 12, 2022
    
    I’m with @josephscott on this one; `async` is error-prone, and I think it should be avoided in the spirit of making it as hard as possible to accidentally shoot yourself in the foot.
    
    I’m not aware of too many legitimate use-cases for `async`, and the ones I’m aware of are all related to dynamic loading scripts after DCL / `load`, which means they don’t apply to this discussion (since this is about what’s included in the initial document). I might be missing others, of course, and if so I’d love to hear about them.
    
    That said, even with any other potential use-cases, as far as I can tell there isn’t much of a pressing need for `async`. And seeing as there are still mechanisms to have it when really needed (through the use of filters, or through the use of handcrafted script tags in PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher), I think the downsides of adding an easy path to `async` outweigh the benefits here.
    - Adam Silverstein 5:44 pm on December 12, 2022
      
      > I think the downsides of adding an easy path to `async` outweigh the benefits here.
      
      Sérgio – thanks for the feedback, I understand and share your reluctance to supporting `async` and “accidentally shoot(ing) yourself in the foot”. Let’s continue to gather use cases and thinking about how we can guide developers towards preferring `defer`. If we have even a couple solid use cases for `async` it is worth considering as long as we clearly document why `defer` is preferable.
      - Sérgio Gomes 11:54 am on December 13, 2022
        
        Thank you, Adam! That sounds good, happy to help revisit this at any point once we have more data 👍
Drivingralle 12:30 pm on December 11, 2022

Would love to see it in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. 👍
Adam Wood 8:22 pm on December 11, 2022

Yes!
jakeparis 2:11 pm on December 12, 2022

We would definitely use this feature immediately. We’d love to remove our “brute force” method of adding script attributes by parsing strings. 🙁
Phi Phan 12:10 pm on December 13, 2022

There is a potential issue with defer and async. The inline scripts added with wp_add_inline_script will run before the scripts that they are depends on. Because it’s inline so there is no waiting time to load as external scripts and we’ll get lots of undefined errors. I got this issue when developing a solution to attach external scripts and inline scripts dynamically for a blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience..
- Adam Silverstein 4:49 pm on December 13, 2022
  
  Good point Phi!
  
  This is why I specifically mentioned consideration of `wp_add_inline_script` for the feature. As you noted, when one script is marked as “after” another it is indicating it depends on execution of that first script, so deferring would cause a problem.
  
  Without deciding on implementation details here, I think we could consider either not deferring the dependent script that you are hooked to (simplest, most reliable) – ie build it into the dependency logic that ensures all scripts in the chain are deferrable, or potentially provide a mechanism to lazy execute the inline script.
- Phi Phan 2:12 am on December 14, 2022
  
  We may have to ignore defer, and async on scripts that have inline scripts depending on them. I don’t think there is a good solution to force the inline scripts to run after the deferred scripts they depend on.
luisherranz 3:34 pm on December 13, 2022

I’m glad to see progress on this. Thank you, Adam 🙂

Are there any ideas for supporting JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. module scripts within this initiative? (ES6 modules using type="module" and the corresponding nomodule attribute)
- Adam Silverstein 5:01 pm on December 13, 2022
  
  > Are there any ideas for supporting JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. module scripts within this initiative? (ES6 modules using type=”module” and the corresponding nomodule attribute)
  
  While not directly part of this approach currently, it is possible a similar idea could by applied to `type`.
  
  Currently I guess you have to use the raw tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. to change the type or add module? How do you think you could declare what module type to use without direct tag control? Could we add a new module attribute, how would that work?
  - luisherranz 9:47 am on December 14, 2022
    
    I don’t know about any specific requirements to support type="module" yet, but I want to start exploring it in conjunction with import maps in the upcoming months, so happy to stay in touch 🙂
    - Greg Ziółkowski 12:20 pm on February 9, 2023
      
      Actually type="module" is a modern version of defer (from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules#other_differences_between_modules_and_standard_scripts):
      
      There is no need to use the defer attribute (see attributes) when loading a module script; modules are deferred automatically.
      
      You can still mix them with async. So maybe we could have also module strategy as an alternative to defer.
      
      The biggest challenge with ES Modules is that they would work only with simple use cases without import maps. Anyway, we could tackle that part separately.
      - Sérgio Gomes 12:40 pm on February 9, 2023
        
        Do note that `type=module` comes with additional cross-origin restrictions; you’ll need CORS headers in place to be able to load and execute a script from a different origin.
        
        Greg Ziółkowski 12:42 pm on February 9, 2023
        
        It’s a good reminder that a simple string for a strategy might not be enough, but we can always use an array in a form like:
        
        array( 'module', array( 'cors' => $cors_config ) )
Greg Ziółkowski 12:37 pm on February 9, 2023

Excellent work putting this proposal together. It’s very exciting to see the discussion going, as we really need to make it easier for developers to use those features. I like how the $in_footer (bool) retains backward compatibility when becoming $strategy (bool|string). Here comes my question, was the intention to allow a single string value for the strategy, or do we want to support an array of strings instead (or both)? A random example that assumes that someone would like to load a script written as ES Module and located at the end of HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. document:

wp_enqueue_script( $handle, $src, $deps, $ver, array( 'module', 'footer' ) );
Weston Ruter 10:59 pm on February 9, 2023

It’ll be nice for comment-reply to no longer be blocking!
Dennis Snell 9:17 pm on February 13, 2023

In the past I’ve used the `script_loader_tag` filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. to made scripts load without blocking the page. Having this available in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. directly would alleviate concerns with that.

On this note, most code I’ve seen in practice wipes out any existing modification to the `script` tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) and so I think it’s going to be helpful having a policy framework available in `_enqueue_script` to avoid that. (Small plug, the Tag Processor is now available to handle half of that equation, which is the need to conveniently and surgically alter the tag rather than wiping it out with `~<script[^>]+>~`).

Noted in this discussion already, `async`, `defer`, and `type=module` are correlated and can mess with each other. We shouldn’t set `defer` if we’re issuing a module.
I think we could consider either not deferring the dependent script that you are hooked to

To me simplicity here is likely more valuable than it seems. If we could draw a line and say, “if you enqueue this script using the HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. script element semantics then you are opting in to the spec-defined resolution performed by the browser.”

I’m thinking through the combinatorial explosion that appears once people realize it’s easier to split and lazily serve all their modules. I don’t want WordPress to perform this graph solving on every page render when the browser is capable and going to do it anyways.

That is, if someone writes a synchronously-loading script or inline script and they call out another module as a dependency that’s loaded outside of the page render, then that’s a clear break in functionality. Supposing we efficiently resolve the dependencies I think there’s a shift in expectations for synchronously- vs asynchronously-loading scripts and resolving the load may still leave code broken.

For example, a library that previously enqueued a script not only updates to add `defer` or `type=module`, but also rewrites the code inside the module to assume an async environment. One scenario that comes to mind is a script looking for a known element in the DOM. Maybe it has to add a `DOMContentLoaded` callback because now it could be possible that _in isolation_ this script evaluates before the Element above it exists on the page. What used to be available immediately after evaluating the script is now only available after an asynchronous wait anyway regardless of how it was loaded or in what order.

That is, are we certain that solving the loading order within WordPress would prevent compatibility breaks and if not, is it worth the extra machinery?

—

Given the needs I’ve had in the past I would think that keeping things shallow to the HTML spec would be beneficial too. There are surely things to come with import maps and other lazy-loading strategies for modules. The less we abstract on top of that model the easier it should be to adopt changes, even if we are left exposing some internal details in the process (those details being things like the actual tag attributes involved).

The nitty-gritty matters here because people are trying to optimize the nitty-gritty.