REST API Roadmap

If you’ve been following WordPress development this year, you may be wondering “what’s been happening with the REST API focus?” We’ve been a little under-the-radar for most of the year so far, so we thought publishing an update and roadmap might be a good idea.

For new contributors looking to get involved with the REST API focus or WordPress generally, there’s never been a better time, and we’d love to have your help on our projects. Read on to see what we’ve been doing, where we’re going, and how you can get involved.

Since 4.7

Since the REST API was merged into core in WordPress 4.7, development activity has unfortunately been light. The merge into core was a huge effort, and after shipping in 4.7 we saw a drop-off in contribution and overall momentum as many API contributors took a break to recover from the stress of the merge. These contributions have not returned to previous levels, and there’s a few factors behind this: the move to Trac, lack of a forward roadmap, and overall fatigue have hampered our ability to move forward quickly.

The core REST API focus goal is to utilise the REST API within the WordPress admin. Defining the scope of this project has involved auditing all admin-ajax calls, as well as the filters used inside these calls, and where they are used. In addition, we’ve been working on the low-level JavaScript utilities we need to offer conceptual compatibility: while we can deprecate and remove old PHP filters, we need to offer new JS-based filters to replace them.

The admin-ajax audit revealed that the majority of ajax requests can be grouped into four categories: Media, Themes, the Editor, and List Tables. The code for both Media and the Editor will gradually switch to the REST API moving forward with development around Gutenberg, and likewise endpoints for better managing Themes are expected to be incorporated into Customizer work.

Rather than simply refactor the existing code for the other actions in a piecemeal fashion, we’ve been working on prototyping bigger groups of related changes and features, starting with the New List Tables and Live Settings.

A significant portion of existing admin-ajax code is for handling list table actions. The existing JS code for these actions is particularly difficult to work with, and the existing list table actions user experience is frustratingly inconsistent (for example, deleting a comment happens inline, whereas deleting a post causes a page refresh). A reworking of the code has the potential to improve UX significantly. New List Tables allows us to explore ideas around how we can improve the content organisation and management experience in the admin. This is a prototyping plugin where we’ve been exploring backwards compatibility techniques, and thinking about how a theoretical new management interface would look.

The Live Settings prototype uses the REST API settings endpoints to add live saving to the Settings screens. This dovetails with the work underway by the Accessibility team to switch to the Settings API, and the two projects will be able to work together in the future.

In addition to these API team projects, work has continued on REST API-related pieces on other teams, notably the Customizer and Multisite teams, who are working on API endpoints in their respective components.

Renewing our focus

Moving forward with the REST API, there’s a few key items we’re going to be focussing on. These items will have their own dedicated subteams and development cycles, and will work in parallel. The two broad goals are to use the API in the admin, and to solve authentication for external applications.

API in the Admin

Getting the API used in the WordPress admin is our primary focus. While it is technically possible to directly switch from admin-ajax calls to REST API calls, this is essentially refactoring with no real user benefit. Instead, we want to focus on changes that can improve the user experience.

For the feature prototypes (New List Tables, and Live Settings), we’re engaging members of the Design team to lead these features from a UX perspective. So far, these prototypes have been primarily about proving out the features and ensuring it’s actually technically possible to migrate these features to use the REST API; with the initial success on the technical side, we need to switch focus to delivering compelling user experiences.

New List Tables will be working with the goal of prototyping improved content management, using the REST API. This includes unifying and standardising existing interactions, and improving the perceived performance. This is a React-based prototype, and uses the existing REST API endpoints.

Live Settings will be working with the goal of making settings changes seamless. We’ve seen huge strides forward with the Customizer for updating your site, and the backend settings deserve a better experience to match. Live Settings touches on similar areas as the Settings API Enhanced project spearheaded by the Accessibility team; we plan to continue working independently to avoid blocking each other, while keeping in touch about the respective projects.

Work on converting existing admin-ajax code to use the REST API will continue, however this won’t be a priority, as it generally doesn’t provide a strong benefit to end users. Most admin-ajax actions will naturally be deprecated as part of progress by other focuses, including Gutenberg and plans around the Media Library. We’ll continue working with other teams and focuses on their efforts here.

Authentication

External authentication is an unsolved problem, and one that’s crucial to API use outside of WordPress core itself, including the official WordPress apps.

There are two key problems to solve here: how do apps act on behalf of a user (authentication), and how do sites recognise valid apps (initial connection). We have existing solutions to both these problems (OAuth 1.0a and the Broker system respectively), however these are not the easiest solutions, and aren’t adequate for all use cases.

To date the most complete authentication solution maintained by the REST API team has been a plugin providing OAuth 1.0a authentication. Moving forward, we are switching authentication focus to OAuth 2. As Matt announced last year, we are going to begin shipping HTTPS-only features in WordPress: this allows us to switch to OAuth 2. Work started during the WordCamp Europe contributor day on a new official OAuth 2 provider plugin which is now under ongoing development.

Simplifying the initial connection is a much harder piece, and this is a long-term project. Eventually, this should be as simple as a “Connect to WordPress” button, requiring minimal effort for app developers and no effort for users. This is a complex problem to solve, and no similar software has to work on the same scale we have. In the meantime however, we’re going to investigate pre-configuring Core to recognize and permit authentication from certain default apps, including the official WordPress mobile applications. Whitelisting applications in core is a practical expedient but this solution is not sustainable in the long-term, and should be replaced with a better system as soon as feasible. Work on solving this issue will be in the mid-term, however, as we need to ensure we have solid basics first.

Help Us!

The toughest challenge facing the REST API team right now is resourcing. There are only a few people working on the API regularly, and we need help to build out our projects—which is hopefully where you come in. We need people of all skillsets to help on New List Tables, Live Settings, and OAuth 2. This includes regular contributors, JS developers, and designers. And all of this will need documentation, too: following a productive contributor day at WordCamp Europe we are making progress expanding and reorganizing the REST API developer handbook, and would gladly welcome any interested docs contributors.

Our plan is to release the first public beta of each of these projects within the next month, with regular releases from each project following that. New List Tables and Live Settings could be part of either a 4.9 or 5.0 release, while OAuth 2 will remain as a plugin until fully proven out, and would likely target a core release next year. This also requires coordination with the Mobile team, and finalising the approach to usage inside the apps.

If you’re interested in getting involved, we’d love to get your help. The API holds weekly meetings every Wednesday at 13:00 UTC (next meeting at Wednesday at 13:00 UTC), and we’re always happy to spend time helping people get started contributing.

#rest-api, #roadmap

Outlining a possible roadmap for the Customizer

Planning for the future is a necessary and important part of the WordPress development process. As we consider the future of WordPress – both as a whole and individual features – we publish proposed roadmaps to encourage greater discussion and give insight into the core team’s thought process.

The process of creating a roadmap is just as important as the vision behind it and the final roadmap itself. This process gives the entire community an opportunity to research and document history, define what specific items can be accomplished to bring us closer to the vision, and outlines how those tasks fit together within a possible timeframe.

What follows is a potential roadmap for the Customize component. If you’re interested in the future of live preview in WordPress, now is the perfect time to get involved and leave your feedback.


A couple of months ago, the WordPress lead developers met with the maintainers of the Customize component to discuss the future of live preview in WordPress. The goal of the chat was to come up with a potential roadmap for both the component and for how live preview can improve the user experience of WordPress for all users.

The ultimate goal of live preview in WordPress is to create user trust and remove the “save and surprise” inherent in some of the backend features.

After a lot of discussion, the group decided to target the following goals over the next two years:

  • Considerably improve performance.
  • Continue iterating on current live preview features to ensure they are solid and as easy-to-use as possible, including theme browsing and installation, menus, and widgets.
  • Experiment with new and different user interfaces. If we were creating live preview today, what would it look like? In what ways can we ease the feeling that you’re looking through a “porthole”?
  • Removal of the ambiguous mode. Currently, the Customizer is contained in a sidebar without the admin toolbar, but ideally there is the admin and the theme, and no in-between. One direction this may go is enabling “Customize” on the front end to immediately load the Customizer controls.
  • Experiment with a guided new user experience (NUX). Live preview lends itself to site setup. How can we improve the live preview experience and combine it with the NUX? Consider a “setup wizard” use case and ensure the flow has no dead ends, i.e. users can customize everything in one.

Those overall goals for live preview in WordPress can be rewritten into some specific features that are in development or planned for the future of the Customize component. These include:

  • Transactions. This re-architecture of some of the Customizer internals improves compatibility with themes by loading the preview using a natural URL, and allows Ajax requests or even REST API requests to be previewed. It also allows the preview to be viewed independently of the Customizer, so changes can be shared for others to review. See #30937.
  • Selective refresh. Only a piece of the page will need to be refreshed when this backend feature is implemented. (Formerly known as “Partial Refresh”.) Currently, this is available for menus in the Customizer. This eliminates duplication of display between PHP and JS, keeping it DRY. See #27355.
  • Concurrency. Allows for “locking” settings using the Heartbeat API, improving the overall user experience by preventing users from overwriting each other’s changes. See #31436.
  • Revisions. Enables plugin developers to add features like draft, roll back, and scheduled changes (e.g. “change my background on January 1”). This builds upon transactions, as the setting changes are staged in a transaction, and this facilitates settings to be revisioned and for settings to be scheduled. See #28721, #31089.
  • Theme installation. Iterates on and completes the theme browsing experience.
  • Responsive preview. Iterates on the concept of live preview by giving users a better idea of what their site will look like on other devices. See #31195.
  • Bootstrapped Customizer. Lazy-load the Customizer into the current frontend view without having to leave the page. With selective refresh implemented, inline controls and frontend bootstrapping would be possible since full-page refreshes would no longer be required.
  • Improvements for both touch and small devices.

Beyond those features, the group identified some specific changes that should be prioritized, in conjunction with the features planned:

  • The sliding animation between panels should feel more like “moving panels” (see: iOS).
  • Keyboard navigation should be consistent and clear.
  • Identify “dead ends” in the interface and remove them, when possible. For example, prior to menus in the Customizer, it was not possible to customize that aspect of your site’s design with the Customizer.

The concepts surrounding live preview and the Customizer have been in development for a long time. Many of the concepts from Elastic Theme and the Visual CSS Editor have been incorporated over time. Over the next few years, experimentation with these concepts will likely take place in feature plugins. For example, this team may experiment with inline content editing, where it makes sense in the context of customizing a site. Another path for exploration is simple theme customization – e.g. change the header font, change the sidebar color, or change the width of the sidebar.

As with all components and new features, we shouldn’t be afraid to experiment and fail and should continually push for new experiments and ideas, especially in the context of feature plugins. Further, some of the above experiments may not make it into core, but are meant as a general direction that live preview should take in WordPress.

Taking these features together, below is a sequence outlining a possible roadmap for live preview and the Customize component in general, along with estimated targets. Please note that this is a proposed roadmap and is entirely dependent on contributor involvement. Additionally, many of these things will take place in a feature plugin prior to core inclusion.

  • Partial refresh. Performance Improvements. (Target: 4.4)
  • Responsive Preview. Transactions. (Target: 4.5)
  • Concurrency. Revisions. Theme Install. Beginning of NUX wizard. (Target: 4.6)
  • Focus on touch screen / small device improvements. (Target: 4.7)
  • Developer API improvements based on feedback from plugin developers. (Target: 4.8)
  • Improved UI after experiments in 2016. NUX “wizard mode.” (Target: 4.9)

Live preview is one of the most critical features in WordPress as we continually combat “save and surprise.” The Customizer in its current form provides an improved user experience to WordPress users when customizing their site’s design. Each feature mentioned above is a continuation of the live preview concept, building and improving upon the Customizer.

Everything above is just a proposal and we need your feedback to ensure it is the right direction. If you’re interested in any of the above, comment here with your feedback, or join the team in #core-customize.

This post was a collaboration between @helen, @nacin, @mark, @celloexpressions, @samuelsidler, and yours truly.

#customize, #roadmap, #roadmaps

An update on shared term splitting

Background reading: #30335 and the notes on shared term splitting in https://make.wordpress.org/core/2014/11/20/dev-chat-summary-november-19th/.

As part of the work on the taxonomy roadmap in 4.1, shared terms (terms which are shared across more than one taxonomy due to a slug collision) now get split when the term gets updated. The end result being that editing a term in one taxonomy will no longer affect the term in another taxonomy.

Many plugins store term IDs in post meta, options, terms (!), etc, and the side effect of shared term splitting is that these caches can break when the ID of a term they reference changes. Not only that, but the breakage (and the cause) can be invisible to the user and hard to track down if they do notice it.

Following a discussion in #core Slack I’m proposing that shared term splitting is removed from 4.1 and we’ll add it back in for 4.2 and allow a full release cycle to get the word out to developers about the change. I’ll make the final decision during tomorrow’s dev chat.

Thanks in particular to @mboynes for his time spent on this.

(Note that this only affects existing terms; new terms do not get shared since r30240.)

#4-1, #dev-notes, #roadmap, #taxonomy

Announced the feature freeze and first b …

Announced the feature freeze and first bug hunt.

#25, #bug-hunt, #roadmap

2.5 is now in feature freeze. Bug fixes …

2.5 is now in feature freeze. Bug fixes and polish from now to release.

#25, #roadmap

Considering March 10th as the release da …

Considering March 10th as the release day for 2.5. That allows one month of beta.

#25, #roadmap