Shortcodes roadmap — clarifications

As mentioned in the initial shortcodes roadmap post, the main purpose of this roadmap is to find the best way for improving the shortcodes APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. and moving it forward. Currently it is slow, fragile, and attempts to handle a lot of edge cases. For this, the most important part is:

  • No shortcodes in HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. tags attributes.
  • No HTML in shortcodes attributes.

There are a few things that deserve to be clarified. Simple shortcodes are great. They are easy to understand and be typed directly by the users. Example: [gallery].

Unfortunately many plugins add complex shortcodes with many attributes and often with nested shortcodes. These are a nightmare for the users. They are not intended be typed directly and can be edited by some sort of UIUI User interface. Using shortcodes to store this type of data in post_content is not a good idea. Since there is a UI for entering and editing, it would be better to use a simple shortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. to “hold the place”, and save all the data in post metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress..

Many of these complex shortcodes also include HTML tags in their attributes. To keep that functionality, the second roadmap draft proposed an extended syntax that allows the shortcodes “content” (the text wrapped by [shortcode] and [/shortcode]) to be additionally separated by delimiters. That would allow for shortcode attributes to contain HTML tags that are stored in the shortcode content.

These delimiters are not intended to be typed directly by the users. They are intended for the plugins that have shortcode editing UI and cannot function without storing HTML in shortcode attributes.

At first look this makes the syntax needlessly complex. However after looking at how complex shortcodes are used now, it is relatively the same: these shortcodes cannot be typed directly and are useless without some sort of UI.

There have been questions about line breaks in shortcode content. It is possible to add support for this. However it will benefit only a very small amount of users. Since shortcodes “live” in HTML context, and line breaks are ignored there, typing in the Text editor and switching to the Visual editor will remove all line breaks. Typing in the Visual editor will add paragraph tags. So only users that never use the Visual editor and have to type long, complex shortcodes will see some benefit.

The Shortcode API Roadmap meeting is in #feature-shortcode today at 17z, which is 2015-10-14 1700.

#meeting, #roadmaps, #shortcodes

Shortcode Roadmap Draft Three

This is the third draft of the ShortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. Roadmap. It describes in broad terms the changes that might take place across versions 4.4 through 4.6. This roadmap gives notice to pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party developers that significant changes in plugin design may be needed for compatibility with future versions of the Shortcode API. This roadmap also identifies steps taken to minimize the impact on plugin developers to allow most plugins to continue working with only small changes.

This roadmap is based on decisions made at meetings, feedback received on previous posts, and consultation with the coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. developers.

Our need for a roadmap arose from specific problems in the old code.  There are performance problems in parsing shortcodes, and we need to fix those problems with backward compatibility in mind.  Recent security patches illustrated the problem of not being proactive about security hardening.  Bloat in content filters is another big problem that complicates efforts to correct problems.

Please comment on this new draft.  We will have another meeting Wednesday at 17Z, which is 2015-10-14 1700.

4.4 – New Restriction on Shortcode Names

There is only one item on the 4.4 Milestone. It helps us move toward our goal of security hardening without breaking websites.  Names of registered shortcodes will be slightly restricted by disallowing angle braces.  It should be possible to implement this change immediately with no impact on existing content or plugins.

The < and > characters will be no longer allowed in the $tag parameter of add_shortcode(). Starting in 4.4, attempting to register an invalidinvalid A resolution on the bug tracker (and generally common in software development, sometimes also notabug) that indicates the ticket is not a bug, is a support request, or is generally invalid. shortcode name will result in a helpful error message.

These characters will be forbidden in shortcode names: [ ] < > / &

Also forbidden are the “non-printing characters” including spaces, tabs, new lines, and other control sequences.

Continue reading

#meeting, #roadmaps, #shortcodes

Shortcode Roadmap Draft Two

This is the second draft of the ShortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. Roadmap. It describes in broad terms the changes that might take place across versions 4.4 through 4.7. This roadmap gives notice to pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party developers that significant changes in plugin design may be needed for compatibility with future versions of the Shortcode API. This roadmap also identifies steps taken to minimize the impact on plugin developers to allow most plugins to continue working with only small changes.

This roadmap is based on decisions made at the meeting in #feature-shortcode, as well as feedback on previous posts, and consultation with the coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. developers.

Our need for a roadmap arose from specific problems in the old code.  There are performance problems in parsing shortcodes, and we need to fix those problems with backward compatibility in mind.  Recent security patches illustrated the problem of not being proactive about security hardening.  Bloat in content filters is another big problem in itself.

Please comment on this new draft.  We will have another meeting next Wednesday at 17Z, which is 2015-10-07 1700. TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. tickets that were nominated for closure at the last meeting will be closed today, with references to this draft.

4.4 – Introduce Multiple Enclosures

The two items on the 4.4 Milestone help us move toward our goals of security hardening without breaking websites.  A new delimiter in the shortcode syntax will enable plugin authors and users to always place their HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. between the shortage tags rather than inside of them.  At the same time, the names of registered shortcodes will be slightly restricted by disallowing angle braces in shortcode names.  It should be possible to implement both of these changes immediately with no impact on existing content or plugins.

New Delimiter

A new addition to the shortcode syntax along with documentation of how it works will be created in the 4.4 development cycle.  Its purpose is to allow more than one HTML enclosure in a single shortcode. Use of this new delimiter is optional or as needed.

Enclosure Delimiter:  [parent:attr=]

Usage:  [shortcode] Content HTML [shortcode:name=] Attribute HTML [/shortcode]

Example:  [photo link_to="twentysixteen/"] Here is <b>my</b> caption. [photo:media=] <img src="00.twentysixteen-260x300.png" width="260" height="300" /> [/photo]

Formally:
    delimiter   =  begin code-name middle attr-name end
    begin       =  "["
    code-name   =  1*( ALPHA / DIGIT / unreserved / %x80-FD )
    middle      =  ":"
    attr-name   =  *( ALPHA / DIGIT / "-" / "_" )
    end         =  "=]"
    unreserved  =  "!" / "#" / "$" / "%" / "(" / ")" / "*" /
                   "+" / "," / "-" / "." / ";" / "?" / "@" / 
                   "^" / "_" / "{" / "|" / "}" / "~"

In the examples above, the “name” part of the new delimiter works the same way as an attribute name. The main difference when using this new style of attribute (the enclosure) is improved support for HTML inside the value text. One basic reason why this works better is because our HTML filters do not need to understand how to look in the middle of individual shortcode tags. All of the HTML is located between shortcode tags, making the shortcode and HTML codes easy to process separately.

Continue reading

#meeting, #roadmaps, #shortcodes

Shortcode Roadmap Extended Discussion

We saw a great amount of feedback on the first draft of the Shortcode API Roadmap.  The resounding concensus was that the shortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. syntax we already know and love should not be replaced.

Now we need to bring that enthusiasm and more feedback to the first official meeting to help create a second draft of the roadmap.

On Wednesday at 17Z, which is 9 Sep 2015 17:00.

It is scheduled in the #feature-shortcode channel.

In case you can’t make it to the meeting, here are some of the items up for discussion.

Parser Problems

The biggest issue with keeping the BBCode style syntax is that we don’t have a scalable way to make these shortcodes work in PCRE.  The current pattern searches for all registered shortcodes by name, because searching for matching pairs of braces leads to backtrack limitations and segfaults.  If someone has an idea or knows a good library, now is the time to tell us!  With that said, we are also bringing some new proposals of our own.  One theoretical solution should make it possible to preview the first word of each shortcode tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) so that the full search pattern then only includes the names of shortcodes already found in the input, rather than the names of all registered shortcodes.  This is the best idea so far and could be easy to implement.

HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. vs. Shortcode Syntax

We still want to expand the shortcode syntax to allow multiple enclosures.  So which new tags would work best internally and still look nice for users?  We are now thinking something more like this:

[shortcode] HTML [=part2=] HTML [/shortcode]

Let’s also have a brief discussion about preparing for the eventual removal of HTML-in-shortcodes and shortcodes-in-HTML combinations. As we saw in 4.2.3, it is better to plan for this rather than allowing it to become an urgent surprise update.

Version Issues and Breaking Things

We still need to plan out a change of filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. priorities so that shortcodes will be processed first, before paragraphs and curly quotes. How much impact will this have on plugins now that we will be changing the way existing shortcodes work instead of adding a whole new system? Is it adequate to offer paragraphs and curly quotes as an opt-in only feature?

Will we need weekly meetings after this first one?

Are there other tickets or features, such as nested shortcodes, that need to be roadmapped?

#meeting, #roadmaps, #shortcodes

Shortcode Roadmap Draft One (Proposal – Request for Comments)

This is the first draft of the ShortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. Roadmap. It describes in broad terms a new feature set and migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. that might take place across versions 4.4 through 4.7. This roadmap gives notice to pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party developers that significant changes in plugin design may be needed for compatibility with future versions of the Shortcode API. This roadmap also identifies steps taken to minimize the impact on plugin developers to allow most plugins to continue working with only small changes.

The decision to create this roadmap arose from specific needs that are not met by the old code. Our old [ and ] delimiters were easily confused with the way those characters are commonly used in English quotations and sometimes even in URLs. The proposal to use [{{ and }}] instead should allow a better balance between being able to type in the shortcodes and avoiding confusion with any other input. With these more unique delimeters, we will be able to process registered shortcodes more efficiently because we will not have to search for them by name. Unregistered shortcodes will have more consistency because we can find them more accurately.

Old style delimeters also gave no strong indication whether or not a closing tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) was required. The proposal to use }$] as the delimeter of a shortcode with a following closing tag increases the efficiency of regular expressions, because the search for a closing tag will only happen as needed.

Adding the new style of shortcode syntax provides an opportunity to make significant API changes. One of those major changes is to ensure that shortcodes are processed before paragraphs and before curly quotes. This will lead to greatly simplified code in related functions that currently must find and avoid shortcodes every time they run. We also have an opporunity to re-think the way shortcodes are filtered, and to give plugin authors more control over those filters when registering their shortcodes.

4.4 – Introduce New Syntax

A new syntax and documentation of how it works will be created in the 4.4 development cycle. Support for the new syntax will be introduced, allowing plugins to register for extra features. CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. shortcodes will use the new syntax in all new posts. The old syntax will not change. Old posts will not be affected. Initial work on the syntax concept follows.

Proposed New Syntax

Self-Closing:  [{{shortcode}}]

Attributes:  [{{shortcode  attr1="value1"  attr2='value2'  "value3"  'value4'  value5}}]

Enclosing:  [{{shortcode}$] HTML [${shortcode}}]

Multiple Enclosures:  [{{shortcode}$] HTML [${encl2}$] HTML [${encl3}$] HTML [${shortcode}}]

Escaped Code:  [!{{shortcode}}]

Stripped Unregistered Code:  [{{randomthing}}]

Stripped Unregistered Enclosure:  [{{randomthing}$]  Content also stripped.  [${randomthing}}]

Stripped Empty Code:  [{{ }}]

Ignored Orphan:  [{{shortcode}$]

Ignored Orphan:  [${shortcode}}]

Ignored Orphan:  [${encl2}$]

Ignored Context:  [{{shortcode <br> }}]

Ignored Context:  <a href="[{{shortcode}}]">

4.5 – Deprecate Old Syntax

Starting in 4.5, plugins that register shortcodes without declaring support for new features will raise debugging errors to alert developers that support for the old shortcode syntax is ending.

Old posts will continue to work normally while the old syntax is deprecated.

4.6 – Upgrade Old Posts

Old posts will be automatically converted to the new shortcode syntax. The Shortcode API will continue to provide deprecated support for old syntax so that there is no disruption during the conversion process.

The API should be adequately abstracted so that old plugins are not affected by this conversion. However, as the new syntax will not support HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. inside of shortcode attributes, there is no guarantee that every shortcode will work the same way in 4.6 as in earlier versions.

4.7 – End of Support for Old Syntax

Old shortcodes will stop working in 4.7. Plugins that still produce the old shortcode syntax will be ignored by the Shortcode API.

The upgrade to 4.7 will include a second pass of the conversion of old posts so that any old syntax that was added to posts during 4.6 will still get converted.

In 4.6 or 4.7, if necessary, a filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. could be added to automatically convert any old syntax still being produced by old plugins when new posts are created.

#proposal, #roadmaps, #shortcodes

Shortcodes roadmap

The ShortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. is well loved by developers. Thousands of plugins use it for many cool features.

Unfortunately it wasn’t documented well when it was added. Even now the documentation is somewhat incomplete. The API was also very permissive, allowing many unintended user cases.

The result of these early mistakes is that there are plugins which use shortcodes in very unintended ways: mixed with HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. tags, nested several levels deep, with huge attributes, etc.

What are shortcodes:

  • With one word: placeholders.
  • Convenient way to add dynamic content inside post_content at run time.

What shortcodes are not:

  • A way to conceal user input in post_content.
  • A way to store any type of data in post_content. There are better places and methods for that, like post metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress..

Shortcodes “live” in the same context as HTML tags. They should obey the same rules. Also — no interlinking between HTML tags and shortcodes. Think of the [ and ] being equal to < and >.

Both <p title="<b>my title</b>"> and [paragraph title="<b>my title</b>"] should be illegal for the same reasons. Also <p title="[my-span]">. I know the current shortcodes parser mostly supports these, and some plugins use them, but that will probably need to change “for the greater good”.

There is simply no good reason for trying to support mixing of shortcodes and tags with the current parser. These cases take longer time and more resources on every front-end page load. They require much more complex code to sanitize and ensure they are safe to run. If plugins cannot operate without mixing shortcodes and HTML tags, they will eventually have to implement their own placeholders and parsers, and ensure all data is sanitized properly. This will require a lot less time, effort and processing as the plugins would know what to expect.

We’ve been talking about this with @miqrogroove for a while now. There are several very interesting suggestions in his post on the subject: http://www.miqrogroove.com/blog/2015/shortcode-v44/.

We both agree that we need to create shortcodes roadmap, similar to the taxonomyTaxonomy A taxonomy is a way to group things together. In WordPress, some common taxonomies are category, link, tag, or post format. https://codex.wordpress.org/Taxonomies#Default_Taxonomies. roadmap. This will allow us to fix the shortcomings in the Shortcode API and clear the path for future improvements.

#roadmaps, #shortcodes