Editor Technical Overview

As we start looking at the editor from a technical perspective it’s important we identify the main obstacles and requirements we face before we start conjecturing solutions. As @matt wrote before, the editor focus aims to make writing rich posts effortless. This has taken the path of treating a post as being composed of distinct pieces of content called blocks. These pieces should be easy to insert and manipulate, providing rich and contextual interfaces to interact with as you craft a post.

So how do we go about turning this into a reality? Content in WordPress is, fundamentally, HTML-augmented text; that is to say, it has no inherent data structure. This has been a very important aspect of WordPress and a force for the open web—it speaks to the sense of ownership and freedom WordPress gives you, since it’s always easy to get the full content of your publications—yet the lack of structure gets in the way of the goal to treat content as composed from individual pieces. (This reality also became an issue for the development of post formats a few years ago, but I digress.)

It’s relatively easy to add structure, but it’s not trivial to do so in a way that doesn’t harm data integrity, portability, and the cohesiveness of the post_content. So let’s lay out a first requirement:

① Shape of the Data: Portable Text

To ensure we keep a key component of WordPress’ strength intact, the source of truth for the content should remain in post_content, where the bulk of the post data needs to be present in a way that is accessible and portable, while still providing additional structure on top of HTML semantics for our editing tools. Data needs to be praised and respected. This additional structure would hopefully be invisible to the document’s display context, as it ensures the rendered content is viewable in situations that may not be aware of blocks at all. (Think of email clients, RSS, older editor versions, mobile editors, database dumps, etc.)

Storing things separately means post_content becomes either a pointer or duplicated data, which fragments the source of truth since they can get out of sync easily. (A few content block plugins do this by storing structured data in postmeta and pure data in post_content.) On the other hand, storing things together means structure can become gibberish if it’s not formatted properly before display.

How can we then offer users a great experience when creating or manipulating content without sacrificing the spirit of integrity and data reliability that is expected from WordPress? Good representations of the data would also make it easier to develop robust collaboration tools in the future by allowing us to lock things in a more fine grained way. I believe this is important to figure out soon to allow us to prototype quickly, so I’ll follow up with an initial proposal by the end.

Honouring HTML leads to a second requirement:

② Simplicity and Semantics of Representation

Unless we are improving the semantics of the document we should minimize what markup we add to identify a block; for example, avoid adding extra DOM elements or attributes, both for simplicity and standards sake. WordPress has been a champion of web standards, and we should not venture away from this quality. How can we add structure in a way that remains invisible to the output (as meaningful content as possible) but gives the necessary hooks to infer a structured view for editing purposes?

While we discuss how to structure the content to include invisible demarcation of blocks, the aspect of their nature leads to a third requirement:

③ Static & Dynamic Blocks

Blocks can be either static or dynamic. That is to say, some blocks can be stored with all the necessary aspects needed for their display, while others need to be processed before generating their final output (shortcodes, embeds, widgets, etc). This distinction is important because the most common two blocks people will naturally use are text and images. We should not break their clarity as we treat them as blocks.


 # Static
Here’s some text.

# Dynamic
[text id=123] // Pulls "Here's some text" from somewhere.

This conceptual separation of blocks is useful for designing our project, yet they generate abstract complexity which users should not be exposed to, leading to a fourth requirement:

④ Consistent Experience

One of the biggest benefits of blocks is that composing a post becomes more intuitive and reliable. Everything is inserted under the same assumptions; discovering what can be inserted is a natural part of inserting content. To the user all blocks behave in a consistent and familiar way, even if they provide tailored UIs for their controls.

The user should also be able to edit a post in a different system (mobile, REST, older version of core, apps like Mars Edit, etc), even if they lose the advantages of block editing. That’s another reason why post_content as source of truth matters, compared to storing a JSON structure in postmeta. Having things in two places means they can get out of sync depending on what tool you used to edit.

These nuances of data, UI, and display lead to a final and more general requirement, which is understanding the system we’ll be crafting:

⑤ The System

The editor experience has three fundamental aspects to its system: the UI used to manipulate a block; the demarcation of the block; the rendered output of the block. These are all separate concerns, from the tools we craft to edit a post, to the document syntax that holds the data structure, to the way the final output is generated to be displayed as HTML. (With static blocks that last aspect may be of no significant concern since the document doesn’t care about the presence of static blocks, it just displays them.)

Picturing these concerns as connected but fundamentally separate would help us figure out the best design and technical solutions for each stage, while avoiding us taking aggressive moves by coupling expectations. For instance, JavaScript is a natural technology to look at when it comes to the ability to manipulate and interact with content blocks, yet it may not be the best at all when it comes to rendering the final post to a viewer. Avoiding painting the whole flow under the same light should allow us to focus efforts, because we don’t have to change everything.

❶ Coda

As a final coda, and following @joen‘s design exploration, let’s keep in mind that our first goal should be to set up a reliable foundation to allow us to iterate quickly and test assumptions. I propose we focus initially on a few static blocks (text, image with and without caption, quote) to limit scope of the project.

In which ways can we fulfil ① and ② from the above requirements?

Shortcodes fall short in that they are not invisible, they are opaque, not standing to the scrutiny of semantics, and are also painful to parse. Alternatives could be data-* attributes in the HTML elements or custom elements (paving the way for web components, perhaps), yet we need to be careful with adding cruft.

One other possibility is to look at HTML comments as a way to provide explicit demarcation to post_content without affecting the node structure of the document for something that is inherently spurious to the HTML semantics. WordPress already uses comments for the more tag (<!--more-->) and splitting content into pages in a way that has proven to be quite robust.

It could look something like this:

<!-- wp:image -->
<figure>
  <img src="/">
  <figcaption>A picture is worth a thousand words</figcaption>
</figure>
<!-- /wp -->

There are drawbacks, benefits, and implications with each that we should discuss separately. Are there any other possible solutions?

Please, join us in #core-editor if you are interested. We’re holding weekly meetings in Slack, Wednesdays at 19:00 CET.

#editor

Focus Tech and Design Leads

There are three main focuses this year: the REST API, the editor, and the customizer.

For the REST API we’re going to work on getting first party wp-admin usage of the new endpoints, and hopefully replace all of the core places where we still use admin-ajax.

The editor will endeavour to create a new page and post building experience that makes writing rich posts effortless, and has “blocks” to make it easy what today might take shortcodes, custom HTML, or “mystery meat” embed discovery.

The customizer will help out the editor at first, then shift to bring those fundamental building blocks into something that could allow customization “outside of the box” of post_content, including sidebars and possibly even an entire theme.

Each focus will have a tech lead, and a design lead, and I’ll be working closely with each to make sure we’re aligned and moving diligently in the right direction even though we don’t have the normal release hooks. These starting leads will be:

REST API: Ryan McCue and KAdam White.

Editor: Matias Ventura and Joen Asmussen.

Customizer: Weston Ruter and Mel Choyce.

Given there is no set timeline for releases that would normally set a term, these leads are free to bow out at any time they feel they can’t contribute fully and we’ll find a replacement.

You might be wondering what each lead is responsible for. The REST team gave some thought to this for their focus, and this is the list they came up with:

Tech Lead responsibilities:

  • identify and ensure implementation of first-class REST API usage within WP-Admin,
  • replacing/refactoring admin-ajax use.
  • overall REST API architecture
  • infrastructure and endpoint performance
  • security at an infrastructure and endpoint (data-exposure) level
  • improving authentication options and documentation
  • working with the Design Lead to build new, or expand on existing, endpoints
  • working with the Design Lead to address usability feedback for the infrastructure and endpoints

Design Lead responsibilities:

  • usability of endpoints for internal or external clients
  • usability of the infrastructure from the perspective of a API client
  • working with the Editor and Customizer focus teams to collect requirements and gather feedback
  • identifying ways to improve the overall experience for folks building clients or consuming endpoints (like documentation)

#customizer, #editor, #rest-api

Idea: Uniform Resource Identifiers as an Alternative to Shortcodes

The idea is that any objects embedded in a post’s content have their own home and should be seen as separate resources. Data would be stored elsewhere.

Examples

  • https://example.com/resources/contact-forms/email-ella/
  • https://example.com/resources/charts/php-versions-wordpress/
  • https://example.com/author/iseulde/
  • https://example.com/galleries/yummy-chocolates/
  • https://example.com/surveys/wordpress-editor-2016/
  • https://external.com/some-map-or-form/

Object Types

This approach could be good for:

  • Forms such as contact forms, surveys and polls.
  • Visualised data such as diagrams, charts, graphs and tables.
  • Lists of items such as a galleries, playlists and lists of posts.
  • Albums or manually composed collections of items where the presentation is uniquely different from a normal list.
  • Any content that needs to be reused or organised separately. Anything that can be re-sourced. 😉
  • Any external resources.

This is not good for:

  • Layout.
  • Text conversions.

Inspiration

  • External embeds work similarly in WordPress through oEmbed.
  • Images, audio and video are embedded by their URL in HTML.
  • Media have their own “attachment” post type in WordPress.
  • Many plugins already have a separate post type to store their data.
  • I’ve seen some news media do this already for things for like graphs (post, resource).

Benefits

  • A URL (or “link”) is a concept that is already familiar to many users.
  • URIs are designed for these sort of things. Think about images — it would follow the same paradigm.
  • The content is treated as a separate resource, and can be reused, just like shortcodes, but it forces separation.
  • WordPress allows you to create “pretty” semantic URIs, so the resource can be described for a better experience.
  • A cool side effect is that others could embed these easily on their site through oEmbed. WordPress already supports this.
  • If there’s a problem, the URL will act as a fallback.

Implementation

A quick way to implement this for WordPress 4.4 and higher is registering a new post type. This will handle most things, you just need to provide a custom embed template with the template_include filter. An external resource can be filtered with the Embed API and TinyMCE View API, even if it doesn’t support oEmbed.

Challenge: Variants and Settings

Either each variant of an resource needs its own ID, or settings could be passed with a query string. I think the use of settings should be minimised — for example, columns for a gallery object may be better set per ID. Settings can certainly be useful for things like autoplay, for example YouTube allows you to set the start time.

Challenge: Alignment

This is great if the URL is added on a separate line, but aligning the object is not evident. This is a challenge for shortcodes too. At the moment the core galery shortcode does not allow you to allign it, and the caption shortcode has an attribute for it. Similarly the URl could have a query parameter for alignment, but that doesn’t sound ideal. Alternatively the paragraph could be floated, or it needs to be wrapped in a `div` element to float mid-text. Another approach is to always wrap the URLs in a (custom) tag that can have display information. Again, think about how images are embedded in HTML.

Challenge: Namespace

Shortcodes would have a similar problem, though slug clashes are probably more likely. Ideally plugins should use their own prefix, but this may be seen as ugly. Another way to avoid clashes with other post types is a sub type for “attachments” or “resources”.

Challenge: Extra Queries

I don’t see this exactly as a challenge, as it’s the nature of the concept, but it may be used as an argument against it. Many shortcodes already make use of custom post types to store data and embedding media (or anything else) also requires extra queries. If and how this should be cached is another problem.


I would love to hear your thoughts on this alternative, especially from those who use shortcodes for this type of objects in the post content. If you already use a custom post type, why not take advantage of embedding instead of creating shortcodes? If you want to embed external resources, why wrap it in a shortcode?

If you have other alternatives, I’d love to hear about those too!

#editor, #media, #plugins, #shortcodes

Editor changes in 4.7

There are a few noteworthy changes to the editor in WordPress 4.7.

Some of the toolbar buttons have been rearranged to make them easier to access and to encourage proper use of the HTML elements they insert.  The headings drop down is now moved to the top row, and the strike-through and horizontal rule button are moved down. This also reflects their usage.

The underline and justify buttons have been removed from the bottom row. Underlining is a bad practice as readers can confuse it with links (bad accessibility), and it does not insert a semantic element. Justifying has uneven browser implementation, and in many cases is bad for readability. Keyboard shortcuts for both will keep working.

For more information see #27159 that also has links to the discussions in Slack.

Labels for keyboard shortcuts have been added to the tooltips for buttons and inside drop downs to make them easier to discover.

As always, feedback is welcome.

Ella and Andrew

#4-7, #dev-notes, #editor, #tinymce

Editor changes in 4.6

In WordPress 4.6 TinyMCE is upgraded from version 4.3.10 to 4.4.1. There are numerous bug fixes and several new features, most notably a new inline theme (changelog).

The wpview editor plugin (that is responsible for showing gallery, video, audio, and oEmbed previews) was updated to use the TinyMCE API for non-editable elements. This brought some small changes and improvements in the UI, for example “views” are draggable now. On the back-end the wp-mce-view-unbind event was removed as it doesn’t exist in the API. It was intended for cleanup/unloading but was never very reliable. If a plugin needs to unload instance dependent scripts, it can use mutation observer to monitor when the view node is deleted. See #36434 for more information.

wpview remains an experimental API, though with each iteration it is getting closer to being finalized. As an experimental API, breaking changes are expected. As always, please test your plugin now if it modifies or depends on the editor, especially if you use experimental features like wpview.

#4-6, #dev-notes, #editor, #media, #tinymce

Editor wish list for 4.6 and beyond – chat summary

This list builds on the previous wish list. You can read the full chat in the archive.

Features

  • Allow suggestions and comments to be made, similar to what ICE does. This would make a good plugin and feature project first. @eric and @azaozz seem to be interested in working on something like this.

Enhancements

  • (Publish) meta box revamp. See #36474. Feedback welcome! @michael-arestad, @melchoyce, @helen@mapk@hugobaeta?

    Proof of concept.

    Proof of concept.

  • More experimenting with inline toolbars (separate for formatting and inserting). We could first do this on small screens where the toolbar would be fixed at the top, later maybe on big screens. See also #29923.
  • Could editor scrolling be improved (e.g. hide on scroll down)? See #36482, and also #31751.
  • Caption placeholder. Focusing on an image would give you a placeholder for a caption. See #32175.
  • Leaving dialog. Offer WordPress UI on leaving the page if we can with the option to save changes. See #28566.
  • Advanced panel for the inline link toolbar, so plugins can add options. See #36312.
  • More formatting shortcuts (code blocks, bold, italic…)? See #36433, and also #6331.
    Decide whether to add bold and italic shortcuts at all, how to do the triple back tick shortcut.
    Try to merge with TinyMCE’s own textpattern plugin.
    Any other things we could automatically format in the editor? Curly quotes? Thinking about wptexturize().
  • Save and update without a page reload. For this we will need to look into nonce refreshing. See #7756.
  • Autosave in the browser revamp and improvements. Add some subtle, always present UI for restoring a post from in-browser autosave. Try to better detect when a restore may be needed (and show the current notice). See #36479.
  • Handling nested shortcodes. See #30094. I’m skeptical about this one, but please do let us know what your thoughts are if this interests you.
  • Save custom colours. See #31479.

Under The Hood

  • Consider the new non-editable TinyMCE plugin for our non-editable views. See #36434.
  • Consider the TinyMCE API for inline toolbars, see #36480.
  • Responsive images for TinyMCE. See #36475. Depends on whether we will be saving the srcset and sizes attributes in post content, @joemcgill?
  • Handle inline image blobs in TinyMCE.
  • Different placeholders for more and nextpage tags. See #29804.
  • Auto embed bug. See #25387.

Call for Contributors

If you would like to see more of these features implemented sooner, join us. Everybody can contribute as designer, UX expert, developer, tester, the area you feel most comfortable with. Describing your workflow, how you use the editor, and what you find difficult or easy is also a very good way to contribute.

You can also leave feedback on the relevant tickets.

Let us know if you have more suggestions or ideas to add to the list.

The next chat will be Wednesday, 13 April, 18:00 UTC in #core-editor.

Andrew and Ella

#4-6, #editor

Editor chat 4.6

This Wednesday, 6 April, 18:00 UTC we’ll have our weekly editor chat in #core-editor. This time we’d like to discuss the roadmap and ideas for 4.6, so please join us if you can and are interested in pushing the editor forward. If you can’t attend feel free to comment here, or on the summary of the chat that we will post on this blog afterwards.

#4-6, #editor

The Editor in WordPress 4.5

Inline Link Toolbar

From WordPress 4.5 you will be able to link text with an inline toolbar, which replaces the link modal. You will still be able to access the modal with the gear icon in the toolbar, if you’re using one of the advanced fields or are using a plugin that extends the modal. Eventually we will try to move all those fields inline, though still under an advanced toggle and not visible by default. For those interested, see #36312.

Inline link toolbar.

Text Patterns

We also added some more text patterns, or shortcuts if you like: `text` will change to <code>text</code> and --- (or more dashes) will change to <hr> while typing. We considered adding patterns for bold and italic, but there was no consensus yet and we’d like to test the inline text patterns first on an HTML tag mostly used by developers. See #33301.

#4-5, #dev-notes, #editor

Link modal (wpLink) changes in WordPress 4.5

There is a new and improved inline links dialog in the Visual Editor, see #33301. When the users type in the URL field, it uses jQuery UI Autocomplete to search for local posts and pages.

The old modal dialog (a.k.a. wpLink) is still used for “Advanced” link options. It was simplified, the infinite scrolling bottom part was removed. Now this dialog also uses UI Autocomplete on the URL field to search for posts and pages. That makes it consistent with the inline dialog and leaves more space for plugins that want to add additional settings in it.

If your plugin uses or extends wpLink, please test it now to confirm all is working properly.

#4-5, #dev-notes, #editor

Editor wish list for 4.5

#4-5, #editor