Shortcode Roadmap Draft Three

This is the third draft of the Shortcode API Roadmap. It describes in broad terms the changes that might take place across versions 4.4 through 4.6. This roadmap gives notice to plugin developers that significant changes in plugin design may be needed for compatibility with future versions of the Shortcode API. This roadmap also identifies steps taken to minimize the impact on plugin developers to allow most plugins to continue working with only small changes.

This roadmap is based on decisions made at meetings, feedback received on previous posts, and consultation with the core developers.

Our need for a roadmap arose from specific problems in the old code.  There are performance problems in parsing shortcodes, and we need to fix those problems with backward compatibility in mind.  Recent security patches illustrated the problem of not being proactive about security hardening.  Bloat in content filters is another big problem that complicates efforts to correct problems.

Please comment on this new draft.  We will have another meeting Wednesday at 17Z, which is 2015-10-14 1700.

4.4 – New Restriction on Shortcode Names

There is only one item on the 4.4 Milestone. It helps us move toward our goal of security hardening without breaking websites.  Names of registered shortcodes will be slightly restricted by disallowing angle braces.  It should be possible to implement this change immediately with no impact on existing content or plugins.

The < and > characters will be no longer allowed in the $tag parameter of add_shortcode(). Starting in 4.4, attempting to register an invalid shortcode name will result in a helpful error message.

These characters will be forbidden in shortcode names: [ ] < > / &

Also forbidden are the “non-printing characters” including spaces, tabs, new lines, and other control sequences.

Please note that these limitations were documented over a year ago in Codex and should have no impact on existing plugins. Before 4.4, these were implicit restrictions. The API had allowed any character in the name of registered shortcodes, even when the shortcode parser could not recognize them.

4.5 – Change of Filter Priorities

There is only one item on the 4.5 Milestone. It helps us move toward our goals of simplifying display filters and fixing old bugs.

Reducing complexity of content filters is a major goal for this roadmap.  In reviewing the current API, we find entire extra functions and highly complex regex patterns that are needed only because we are running several filters before processing shortcodes.

The first step in correcting the priority problems would be reducing the do_shortcode filter priority from 11 to 9, causing it to run earlier than most other filters.

A major concern related to changes here is the impact of those other filters after the shortcodes are processed.  Development of the 4.5 API will follow several objectives to make this as seamless as possible.

Curly quotes, known as the wptexturize() function, will avoid shortcode output by default so that we do not cause unexpected changes for plugins.  This is very easy.  Wptexturize already includes HTML element avoidance logic.  We will internally create unique HTML tags or a similar placeholder system so that shortcodes remain unaffected after the change of filter priorities.  At the same time, we will be able to remove the extra code that was needed to look for shortcode tags.

New options will be added for shortcode registration.  Ideally, shortcodes will be able to turn on and turn off the curly quotes feature for each registered shortcode name.

Paragraphs, known as the wpautop() function, will avoid shortcode output by default for the same reasons.  The big difference here is that wpautop already has several known bugs related to code avoidance, and does not yet have the needed logic.  So this will be the larger challenge of the 4.5 Milestone.  It also creates great opportunities for us to focus on some old tickets and to improve this core code along with the shortcode system.  Shortcodes will be able to turn on and turn off the paragraphs feature for each registered shortcode name.

4.6 – HTML Attribute Parsing Becomes Opt-In Only

There is only one item on the 4.6 Milestone. It helps us move toward our goals of security hardening and improving performance within the shortcode display filter.

Shortcodes that appear inside of HTML attributes, such as <a href="[myurl]">, require extra processing. This extra processing is currently performed in many situations where it would not be necessary if these shortcodes were not allowed. However, we don’t want to simply break these shortcodes.

In 4.6, it would be necessary to register an additional option with each shortcode name that needs to be used inside of HTML attributes. The benefit to the majority of websites, which do not use such shortcodes, is a quick skip of the extra processing. For the websites that do use such shortcodes, the extra processing can still be skipped for each post that doesn’t use these specific shortcodes.

Plugins would have the registration option available in 4.5, and it would be required in 4.6. Plugins not updated by 4.6 would only work when used outside of HTML attributes.

Unresolved Issues

Due to lack of traction for shortcode syntax improvements, continued core support for HTML in shortcode attributes is likely to evolve through the automatic update system when unplanned future patches are released.

Ticket #23307 requires additional discussion. The return value from shortcode attribute parsing has some strange characteristics. This should be improved, if it is possible without breaking plugins. Otherwise, there is no foreseeable problem with leaving it unchanged.

One of the questions raised at a previous meeting was whether or not we should place even more restrictions on shortcode names? This needs further discussion.

#meeting, #roadmaps, #shortcodes