WordPress Importer Redux

Hi, I’m Ryan McCue. You may remember me from such projects as the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/..

I’m here today to talk about something a bit different: the WordPress Importer. The WordPress Importer is key to a tonne of different workflows, and is one of the most used plugins on the repo.

Unfortunately, the Importer is also a bit unloved. After getting immensely frustrated at the Importer, I figured it was probably time we throw some attention at it. I’ve been working on fixing this with a new and improved Importer!

If you’re interested in checking out this new version of the Importer, grab it from GitHub. It’s still some way from release, but the vast majority of functionality is already in place. The plan is to eventually replace the existing Importer with this new version.

The key to these Importer improvements is rewriting the coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. processing, taking experience with the current Importer and building to fix those specific problems. This means fixing and improving a whole raft of problems:

  • Way less memory usage: Testing shows memory usage to import a 41MB WXR file is down from 132MB to 19MB (less than half the actual file size!). This means no more splitting files just to get them to import!
  • Faster parser: By using a streaming XML parser, we process data as we go, which is much more scalable than the current approach. Content can begin being imported as soon as the file is read, rather than waiting for pre-processing.
  • Resumable parsing: By storing more in the database instead of variables, we can quit and resume imports on-the-go.
  • Partial imports: Rethinking the deduplication approach allows better partial imports, such as when you’re updating a production siteProduction Site A production site is a live site online meant to be viewed by your visitors, as opposed to a site that is staged for development or testing. from staging.
  • Better CLICLI Command Line Interface. Terminal (Bash) in Mac, Command Prompt in Windows, or WP-CLI for WordPress.: Treating the CLI as a first-class citizen means a better experience for those doing imports on a daily basis, and better code quality and reusability.

Curious as to how all of this is done? Read on!

Continue reading

#importers

REST API: Welcome the Infrastructure to Core

Hi from the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. team! We’re extremely excited to announce the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. infrastructure has now been merged into coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. as of r34928 (plus a couple of fix up commits we won’t mention). Huzzah!

Sincere thanks to every single one of the contributors, we wouldn’t be where we are today without you. It takes time and effort to produce great things, and it’s impossible to make things great without everyone helping. This has been a truly collaborative effort, and I wish I could do more than just give you props.

(Important note: if you have a 2.0 betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. already installed, you must upgrade to beta 5.)

What’s included in 4.4?

As mentioned in the merge proposal, the API comes in two parts: infrastructure and endpoints. In 4.4, the infrastructure is now available as part of core, while the endpoints continue to only be available in the plugin.

You can think of the infrastructure as an “API construction kit”. WordPress 4.4 will make it possible for everyone to build RESTful APIs in a much easier fashion, which will benefit people building custom APIs for their site. The infrastructure handles the routing, argument handling, JSONJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. serialisation/deserialisation, status codes, and all that other lovely REST stuff.

For client authors, this doesn’t help you much right now. Hold tight, the team is working as fast as we can on the endpoints to get them ready for a future release. In the meantime, you can make sure sites you want to work with have the plugin installed, which isn’t a change from the current state.

For pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and theme authors, you can start building new APIs immediately using the infrastructure now in core. This can start replacing your existing custom admin-ajax endpoints or other bespoke code you already have.

To authenticate with the API, only built-in cookie authentication is available out of the box right now. The OAuth 1 plugin will continue to work with WP 4.4 and the API plugin, as will the Basic Auth plugin for local development.

It’s super easy to get started, and there’s even a guide available to kick-off. (Note: the WP_REST_Controller class is not included in WordPress 4.4.) This documentation will be migrated across to developer.wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ soon.

If you want to access any of the built-in data in WordPress without building it out yourself, you’ll need the endpoints as well. These will continue to be packaged in plugin form, and version 2.0 final will be released to accompany 4.4 before the end of this cycle.

What if I’m using the API already?

If you’re on version 2 of the API, you’ll need to update the API to beta 5 or later before updating to the latest version of core. This new version will use the new infrastructure in core if available, or fallback to a compatibility library otherwise.

Important note: Earlier 2.0 betas (1 through 4) are incompatible with WP 4.4. Your site will fatal error if you don’t upgrade to beta 5 or later. You must upgrade to the latest API to run trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. and to run WP 4.4 when it’s released.

If you’re on version 1 of the API, you won’t hit any fatal errors straight away, but endpoints will stop working with 4.4. We’re still planning on releasing a final version 1 release for compatibility, but now would be a great time to consider migrating forward to version 2. Apart from security releases, version 1 has ceased being actively maintained.

Looking forward for the API

Now that the API is past this first hurdle, it’s important to keep looking forward. Our immediate next step is to improve and polish the endpoints for phase two of our merge. There’s a lot of work still to be done here, so we’d love you to join us on GitHub.

The infrastructure of the API will now be maintained via TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress., so new issues and patches should be sent there instead under the “REST API” component. Issues with endpoints should still be filed on GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/. Don’t worry if you’re not sure; you can file issues on either Trac or GitHub, and they’ll be triaged into the correct place as needed. (It’s more important to make sure the issue is filed in the first place!)

The team wants to keep pressing forward with the API and keep up our rate of progress, if not improve it even further, and we’d love your help. We still need help from content writers on our documentation, designers and developers on our authentication plugins, and developers on the endpoints. If you want to help, we can always use a hand, and we’d love to help get you started. We’re available in the #core-restapi room on Slack, and we’d love to see you at the weekly meeting at Monday 23:00 UTC 2015.

We look forward to continuing to work on the API and getting these endpoints happening. Thanks again to everyone who got us here.

(Then again, maybe a REST API is more of a Shelbyville idea…)

#4-4, #dev-notes, #json-api, #rest-api

WP REST API: Merge Proposal

Hello everyone! This is the post you’ve all been waiting for. 🙂

We on the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. team (myself, @rachelbaker, @joehoyle, @danielbachhuber, and newest member and coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. committercommitter A developer with commit access. WordPress has five lead developers and four permanent core developers with commit access. Additionally, the project usually has a few guest or component committers - a developer receiving commit access, generally for a single release cycle (sometimes renewed) and/or for a specific component. @pento) would like to propose merging the REST API into WordPress core. We’ve been working a while on this, and think it’s now ready to get your feedback.

This is our first iteration of the proposal, and we’re actively looking for feedback. If you have thoughts on the project, or on this proposal, let us know! Only with your feedback can we make progress. 🙂

What is the REST API?

The REST API is a nice and easy way to get at your data in WordPress externally, whether that’s from JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. in a theme or pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party, mobile and desktop applications, or importing and exporting data. The APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. offers up all core data types (posts, terms comments, and users), plus support for metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. and revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision.; we’ve got plans to eventually have access to everything the adminadmin (and super admin) and frontend have access to.

The REST API differs from existing WordPress APIs in that it is explicitly designed from the ground up for modern mobile and browser usage, using the lightweight and widely-supported JSONJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. data serialization format with a modern REST interface. Both of these are already familiar to most developers: JSON is a subset of JavaScript intended purely as a data interchange format, and REST is a set of best practices around HTTPHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands.. Both are supported natively by almost every programming language and platform.

Why do we need a new API?

WordPress already has external APIs: XML-RPC, designed for desktop clients; Atom and RSS feeds, designed for post syndication; and the venerable admin-ajax, designed for Ajax requests in the admin and frontend. These APIs all serve different purposes, but often have a great deal of overlap. In addition, these have all been stretched beyond their original intentions: XML-RPC now contains site management tools, RSS has been extended into the WXR export format, and admin-ajax is the catch-all of any sort of browser-server communication in plugins and themes.

The REST API builds upon the heritage of these APIs to provide better support today for using these, as well as laying the groundwork for expanded use in the future.

XML-RPC is the closest analogue to the REST API in terms of usage and capabilities. Originally designed back in 1998 to allow desktop clients to create and edit posts on blogs, WordPress has extended this with both other specifications (such as MetaWeblog) and with its own proprietary additions. Fundamentally, XML-RPC is built around Remote Procedure Calls (RPC), essentially a way of calling a function externally. It then uses XML to serialize the data for passing back and forth.

Unfortunately, XML serialization can be problematic at times. XML has lots of power, but support for custom entities and DOCTYPEs can cause parsing problems and security attacks, including billion laughs, and XXE exploits. (Currently, WordPress has to disable parts of the XML parser and apply regular expression replacements to be able to parse these safely.) XML is also very verbose, and represents data in a way which doesn’t map easily to programmatic data structures. JSON on the other hand is both concise and well-represented in memory, as it’s based on JavaScript’s native syntax.

The admin-ajax API is also very commonly used in WordPress, albeit typically only by plugins and themes. This is a very lightweight API that essentially acts as a mini-router. Typical usage of this API uses JSON, since it’s a browser-based API, but all usage is completely custom. A lot of the usage of this involves retrieving or updating posts on-the-fly, but due to its nature as simply a framework, these are done in completely different ways. This doesn’t lead itself to extensibility, and requires a lot of duplication every time developers want to get data in or out. We don’t want to replace all of admin-ajax though, since some use cases don’t map exactly: UIUI User interface-related interactions or things like the Heartbeat API are great examples of this.

The REST API can help to supplant these uses. Our aim is to eventually replace the XML-RPC API completely, to act as a secondary import/export format, and to replace most (but not all) uses of admin-ajax. The REST API offers an easier to use interface than the existing solutions.

Why this project?

We’ve been working on this project ever since the first WordPress Contributor Summit back in 2012. Since then, we’ve had lots of feedback from core developers, the community at large, and further beyond in the form of client developers. We believe that the REST API has an immense amount of experience behind it, and plenty of viewpoints have contributed to the project’s direction.

The API has seen great usage in the community, from various mobile apps to large news sites. This usage has helped to battle-test and prove out the API. In the process, we’ve found plenty of bugs, and some security issues. Thanks to this feedback, the API is incredibly stable and secure. (The most recent security bugs we fixed were relatively minor bugs.)

We also designed the API from the ground-up to be part of core, following a core-like mentality to our processes. The API is intended to be both a great feature and a base to build off in plugins. We undertook a significant refactoring and partial rewrite in version 2 to make this extensibility even better. This open process also means that most of the design decisions are documented publicly as we’ve engaged stakeholders to gauge feedback.

We believe these pieces combined make this a fantastic feature for WordPress core, and we hope you all agree. 🙂

What’s the plan?

The plan we’re aiming for is a two part merge of the API. For the first stage, the infrastructure code would be merged into wp-includes and made available for plugins and themes. This is an internal API only, but offers an “API construction kit” for developers to use. For the second stage, the endpoints would be merged, and the API would be enabled for sites by default.

This plan splits the API into two parts, infrastructure and endpoints:

  • Stage One: Infrastructure: The infrastructure is the code responsible for routing requests and handling the “meta” layer of the API, including JSON serialisation/deserialisation, linking, embedding, and REST best practices. This adds a simplified routing layer outside of WP’s rewrites, allowing non-query-var rewrites easily, acting as a base for building APIs inside WordPress.
  • Stage Two: Endpoints: These are where much of the complexity of the API lies, as they’re responsible for mapping data from the external JSON format to the internal data structures, and vice versa. The “business” logic of integrating with WordPress is almost entirely contained within the endpoints. These are the more complex part of the API, as they require using deep APIs in WordPress, and handling security and privacy concerns.

With this plan, we would aim to have the infrastructure merged in 4.4, and the endpoints merged one release later in 4.5.

The slow nature of this plan allows a longer review time on the API for core committers. It also gives extra time for reviewing the endpoints, since they would be delayed one release.

Merging the infrastructure now would allow third-party code to begin using the API to build upon it, including migrating from existing custom code. It would also help to increase developer confidence in the API (as it represents a commitment by the project towards a REST API).

In this plan, the first stage would not include any of the base controllers (such as the posts controller). This may limit the utility of the infrastructure for plugins and themes, however as the endpoints would be merged a cycle later, it’s expected that this wouldn’t have a huge impact.

The infrastructure of the API is approximately 2700 lines of codeLines of Code Lines of code. This is sometimes used as a poor metric for developer productivity, but can also have other uses. (roughly a third of the API codebase), and the endpoints make up the remaining 5500 lines of code.

What would happen after merge?

After merging the REST API into core, we’d spend approximately two weeks partying and celebrating. 🙂

Once we’re done with the parties, one major question would be how we manage the API in the future. The existing management and contribution process via GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ has been extremely successful, as we’ve had 61 people’s pull requests merged into the API. Contribution via GitHub is especially useful for the API, as it’s a heavily developer-focussed project, and is aimed at external (non-WordPress) developers. On the other hand, all other contribution to WordPress is done via SVNSVN Subversion, the popular version control system (VCS) by the Apache project, used by WordPress to manage changes to its codebase. and TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress., so integrating with this process is important for existing developers, as well as core’s general processes. We need to ensure the API is an integral part of core, not a separate project.

Given the team’s experience with GitHub as well as Trac, we can bring the best of both worlds by helping integrate the two. This would also improve contribution for WordPress as a whole, and benefit the whole community. This will be especially important as we encourage more contributions from the wider community (client authors, for example). We think we can make good progress here, and we’d love to try and help improve the process. 🙂

In addition to the project management, there are still further API projects we need to tackle. Authentication is the most important of these, as a huge focus on OAuth and similar would be needed to make the API more useful for external clients. Currently, we haven’t had enough time to spend on this as well as managing the API project, however the API is now reaching a finalised stage, so this should be able to improve quickly. Centralised authentication is a huge part of this, as the regular OAuth registration process is cumbersome for a distributed system like WordPress.

Important note: We don’t believe authentication is required for the API merge, and we treat it as a separate project. The authentication is a completely separate system to the API itself. This is something we’d give more time to in the future, but we want to focus on shipping the API for now.

Let’s go!

This is our merge plan for the API, however it’s not finalised. If you’ve got comments, thoughts, suggestions, opinions, or words of encouragement, let us know. We’d love to know what you think. Thank you all, you’re wonderful, and stay golden.

#feature-plugins, #json-api, #merge, #proposal, #rest-api

WP REST API: Versions 1.2.3 (Security Release) and 2.0 Beta 4

First and foremost: version 1.2.3 of the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. is now available. Download it from the plugin repository or from GitHub. This is a security release affecting sites running version 1.2 or a 2.0 betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. releases.

Security Release

Recently, we were alerted to a potential XSS vulnerability introduced in version 1.2 of the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. related to the JSONP support. This vulnerability also existed in version 2.0. Thanks to Alex Concha (@xknown) for reporting this issue to the team responsibly.

This release was coordinated by the REST API team and the WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. security team. The security team is pushing automatic updates for version 1.2.3, but do not wait or rely on the automatic update process. We recommend sites or plugins that are using either v1.2.x or 2.0 beta releases update the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party immediately.

If you’d prefer not to upgrade, you can instead disable JSONP support through a filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.. For version 1:

add_filter( 'json_jsonp_enabled', '__return_false' );

To disable JSONP on version 2:

add_filter( 'rest_jsonp_enabled', '__return_false' );

If you have a question about the security release, you can find the team in #core-restapi on WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., or you can privately message @rachelbaker, @rmccue, @danielbachhuber, or @joehoyle.

Version 2.0 Beta 4

Alongside the security release for version 1.2, we’re also releasing the latest beta for version 2.0: 2.0 Beta 4 “See My Vest”. You can download this from the plugin repository or from GitHub.

This beta release includes the security fix from version 1.2.3, so we recommend everyone running a version 2 beta update immediately to fix the issue.

As well as the security release, this beta also includes a bunch of other changes. Here’s some highlights:

  • Show public user information through the user controller.

    In WordPress as of r32683 (scheduled for 4.3), WP_User_Query now has support for getting users with published posts. To match current behaviour in WordPress themes and feeds, we now expose this public user information. This includes the avatarAvatar An avatar is an image or illustration that specifically refers to a character that represents an online user. It’s usually a square box that appears next to the user’s name., description, user ID, custom URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org, display name, and URL, for users who have published at least one post on the site. This information is available to all clients; other fields and data for all users are still only available when authenticated.

  • Send schema in OPTIONS requests and index.

    Rather than using separate /schema endpoints, the schema for items is now available through an OPTIONS request to the route. This means that full documentation is now available for endpoints through an OPTIONS request; this includes available methods, what data you can pass to the endpoint, and the data you’ll get back.

    ⚠️ This breaks backwards compatibility for clients relying on schemas being at their own routes. These clients should instead send OPTIONS requests.

  • Update JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. API for version 2.

    Our fantastic JavaScript API from version 1 is now available for version 2, refreshed with the latest and greatest changes. Thanks to Taylor Lovett (@tlovett1), K. Adam White (@kadamwhite) and Nathan Rice (@nathanrice).

  • Embed links inside items in a collection.

    Previously when fetching a collection of items, you only received the items themselves. No longer! You can now request a collection with embeds enabled (try /wp/v2/posts?_embed).

  • Move /posts WP_Query vars back to filter param.

    In version 1, we had internal WP_Query vars available via filter (e.g. filter[s]=search+term). For our first betas of version 2, we tried something different and exposed these directly on the endpoint. The experiment has now concluded; we didn’t like this that much, so filter is back.

    ⚠️ This breaks backwards compatibility for users using WP Query vars. Simply change your x=y parameter to filter[x]=y.

  • Respect rest_base for taxonomies.

    ⚠️ This breaks backwards compatibility by changing the /wp/v2/posts/{id}/terms/post_tag endpoint to /wp/v2/posts/{id}/tag.

As always, we have a detailed changelog as well as the full set of changes if you’re interested.

(Note that while this version 2 beta breaks backwards compatibility, the 1.2.3 security release does not break compatibility with the 1.2 branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch"..)

This release had 11 contributors, and we’d like to thank each and every one of them:

$ git shortlog 2.0-beta3...2.0-beta4 --summary
     1   Daniel Bachhuber
    11   Daniel Jalkut
     1   Fredrik Forsmo
     1   Jared Cobb
     3   Jay Dolan
    26   Joe Hoyle
    10   Josh Pollock
    25   Rachel Baker
    50   Ryan McCue
    24   Stephen Edgar
     8   Taylor Lovett

Thank you again to all of our beta testers, and thanks to everyone who let us know how you’re using the API. We’re taking note of all of your feedback, and you might see some further changes related to that in coming releases.

#feature-plugins, #json-api, #rest-api, #security, #updates

REST API: Who’s Using This Thing?

Hi everyone!

This is a break from your regularly scheduled release posts. We’re looking to gather some feedback on the lead up to merging into coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., to assess what your thoughts are on the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.. Whether you’ve used the API or not, we’d love to hear your thoughts.

Here’s a bunch of questions to start you off:

  • What are you doing with the API?
  • What would you like to do with it? (How can we make it better for you? What can’t you do right now that you need?)
  • What improvements would you like to see? (What sucks?)
  • Is the API important to you?
  • Anything else you’d like to tell us?

We really want to make sure this API fits your needs. Without your support, the API really means nothing, so we want to make sure we get this basically perfect first. We’d love to hear feedback from everyone using this, from JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors.-only developers coming to WP for the first time, through WordPress pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and theme developers, all the way through to PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher developers not involved with WordPress.

If your comment is too long to fit here, or you’d like to wax lyrical about the API, feel free to comment on your own blogblog (versus network, site) and cross-post a link across to here.

If you can’t comment publicly due to disclosure, you can leave a message for me personally at me at ryanmccue.info. Please specify if you will allow me to share your private feedback with the WP REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. project and core team, or if you’d prefer to keep it completely private between the two of us.

Help us make this the best feature to ever land in WordPress. 🙂

#json-api, #rest-api

WP REST API: Version 2.0 Beta 1

Hi there, we’re the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. team. You may remember us from such releases as Version 1.1 “Hammock Hut” and Version 1.2 “S-M-R-T”. Today, we’re excited to announce the first betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. release of version 2 of the REST API: 2.0 Beta 1 “Ralph Wiggum”. Go and grab it from GitHub right now, we’ll wait.

Over the past 9 months, we’ve been reworking some of the important internals of the REST API to improve extensibility for site developers, and usability for both pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party developers and client developers. We’ve codified some of our important internal conventions, added plenty of new tools for you to play with and simplify your development, and reworked some of the less fantastic parts of the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.. We think these changes will make the API a whole lot better for everyone, and we’re excited to start getting your feedback on these changes.

One of the things that you’ll notice immediately when using version 2 is just how much hasn’t changed. There’s a lot of things that we’ve changed under the hood, but the fundamentals of the API haven’t changed up too much. Version 2 is a direct continuation of version 1, so most of the API will be instantly familiar to you if you’ve used version 1 before. We have a primer on the important changes since version 1, including a guide on migrating existing endpoints across to version 2.

Here’s some super important notes on version 2, before you get started:

  • No Backwards Compatibility: Version 2 is not backwards compatible with version 1. We have plans to ensure that a version 1 compatibility layer exists, but we’ve leaving that until the API has settled down fully before creating that. However, porting endpoints is easy if you want to give it a go.
  • No Forwards Compatibility: Beta 1 is also not guaranteed to be forwards compatible with future betas. Although this is a beta release, it’s still likely that parts of the REST API will change. However, at this point, the coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. infrastructure of the API is unlikely to change in a breaking manner, so internal plugin APIs are pretty solid. We reserve the right to break these in future though, so keep that in mind when updating.
  • Development Only: Due to the lack of forwards and backwards compatibility, we strongly recommend you do not run the beta in production environments. While it is possible to run it (both security and privacy have solid, thorough handling), any updates will likely break your site.
  • Run Concurrently: You can easily run both version 1 and 2 side-by-side, since the prefixes for all functions and filters have changed. This will require changing the URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org for version 2 however, which can be done with the following filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.:
    add_filter( 'rest_url_prefix', function () { return 'wp-rest'; } );

With all this said, we’d love you to test the plugin as soon as possible. We’re looking for feedback on every part of the API, especially the changes since version 1. If there’s anything that has made it easier or harder since version 1, let us know. We’re continuing to improve the API every day, and now is the time to let us know what’s important to you. (P.S. Did I mention we have an issue tracker?)

We’re excited for this new stage of the API, and we hope you are too.

#json-api, #rest-api

JSON REST API: Version 1.1.1 (Security Release)

I’d like to announce the availability of version 1.1.1 of the JSON REST API. This is a security release for a minor security issuesecurity issue A security issue is a type of bug that can affect the security of WordPress installations. Specifically, it is a report of a bug that you have found in the WordPress core code, and that you have determined can be used to gain some level of access to a site running WordPress that you should not have., however we recommend all users running 1.1 upgrade as soon as possible.

This release only affects users running WP APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. on a domain with other (non-WordPress) software running. Using the JSONP support built-in to the API, it is possible to serve up arbitrary Flash SWF files from the API, allowing these Flash files to bypass browser cross-origin domain policies. While WordPress includes built-in CSRF protection, other software running on the same domain may not include similar protections.

As a workaround, JSONP support can be disabled on your site with:

add_filter( 'json_jsonp_enabled', '__return_false' );

Thanks to @iandunn for reporting this issue to the team responsibly.

We’d also like to announce that WP-API is now available on HackerOne. We invite security researchers and developers to report any potential security issues to us via HackerOne, allowing us to triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. and fix issues privately, and also award bounties for valid security reports.

#json-api, #rest-api

JSON REST API: Version 1.1

I’m happy to announce the availability of version 1.1 of the JSON REST API.

This release is a bit of a smaller, more focussed release as we work on increasing test coverage and squashing bugs. Here’s the juicy details:

  • Add new routes for taxonomies and terms.

    Taxonomies and terms have now been moved from the /posts/types/<type>
    namespace to global routes: /taxonomies, /taxonomies/<tax>,
    /taxonomies/<tax>/terms and /taxonomies/<tax>/terms/<term>

    Test coverage for taxonomyTaxonomy A taxonomy is a way to group things together. In WordPress, some common taxonomies are category, link, tag, or post format. https://codex.wordpress.org/Taxonomies#Default_Taxonomies. endpoints has also been increased to 100%.

    Deprecation warning: The /posts/types/<type>/taxonomies endpoint (and
    sub-endpoints with the same prefix) have been deprecated in favour of the new
    endpoints. These deprecated endpoints will now return a
    X-WP-DeprecatedFunction headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. indicating that the endpoint should not be
    used for new development, but will continue to work in the future.

    (props @kadamwhite, @rachelbaker, @rmccue, #198, #211)

  • Allow customizing the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. resources prefix

    The API base (typically wp-json/) can now be customized to a different
    prefix using the json_url_prefix filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.. Note that rewrites will need to be
    flushed manually after changing this.

    (props @ericandrewlewis, @rmccue, #104, #244, #278)

  • Give null as date for draft posts.

    Draft posts would previously return “0000-00-00 00:00:00” or
    “1970-01-01T00:00:00”, as draft posts are not assigned a publish date. The API
    now returns null where a date is not available.

    Compatibility warning: Clients should be prepared to accept null as a
    value for date/time fields, and treat it as if no value is set.

    (props @rmccue, #229, #230)

  • Fix errors with excerptExcerpt An excerpt is the description of the blog post or page that will by default show on the blog archive page, in search results (SERPs), and on social media. With an SEO plugin, the excerpt may also be in that plugin’s metabox..

    Posts without excerpts could previously return nonsense strings, excerpts from
    other posts, or cause internal PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher errors. Posts without excerpts will now
    always return an excerpt, typically automatically generated from the post
    content.

    The excerpt_raw field was added to the edit context on posts. This field
    contains the raw excerpt data saved for the post, including empty
    string values.

    (props @rmccue, #222, #226)

  • Only expose email for edit context.

    User email addresses are now only exposed for context=edit, which requires
    the edit_users permission (not required for the current user).

    The email address field will now return false instead of a string if the
    field is not exposed.

    (props @pkevan, @rmccue, #290, #296)

  • Correct password-protected post handling.

    Password-protected posts could previously be exposed to all users, however
    could also have broken behaviour with excerpts. Password-protected posts are
    now hidden to unauthenticated users, while content and excerpts are shown
    correctly for the edit context.

    (Note that hiding password-protected posts is intended to be a temporary
    measure, and will likely change in the future.)

    (props @rmccue, #286, #313)

  • Add documentation on authentication methods.

    Full documentation on authentication
    is now available. This documentation explains the difference between the
    various available authentication methods, and notes which should be used.

    (props @rmccue, #242)

  • Include new client JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors. from github.io

    The WP-API JavascriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. library is now loaded dynamically from
    wp-api.github.io to ensure it is always up-to-date.

    (props @tlovett1, #179, #240)

  • Don’t allow setting the modification date on post creation/update.

    As it turns out, WP coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. doesn’t allow us to set this, so this was previously
    a no-op anyway. Discovered during test coverage phase.

    (props @rachelbaker, @rmccue, #285, #288)

  • Check post parent correctly on insertion.

    Posts could previously be added with an invalidinvalid A resolution on the bug tracker (and generally common in software development, sometimes also notabug) that indicates the ticket is not a bug, is a support request, or is generally invalid. parent ID. These IDs are now
    checked to ensure the post exists.

    (props @rmccue, #228, #231)

  • Make sure the type is actually evaluated for json_prepare_${type} filter.

    This value was previously not interpolated correctly, due to the use of the
    single-quoted string type.

    (props @danielbachhuber, #266)

  • Return WP_Error instead of array of empty objects for a revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision.
    permissions error.

    Previously, when trying to access post revisions without correct permissions,
    a JSONJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. list of internal error objects would be returned. This has been
    corrected to return a standard API error instead.

    (props @rachelbaker, @tlovett1, #251, #276)

  • Flip user parameters check for insert/update.

    Previously, you could add a user without specifying username/password/email,
    but couldn’t update a user without those parameters. The logic has been
    inverted here instead.

    (props @rmccue, #221, #289)

  • Add revision endpoints tests

    (props @danielbachhuber, @rachelbaker, @rmccue, #275, #277, #284, #279)

  • Add post endpoint testing

    Now at >54% coverage for the whole class, and >80% for the main methods. This
    figure will continue to rise over the next few releases.

    (props @rachelbaker, @rmccue, #99)

  • Separate helper functions into global namespace.

    WP_JSON_Server::get_timezone(), WP_JSON_Server::get_date_with_gmt(),
    WP_JSON_Server::get_avatar_url() and `WP_JSON_Server::parse_date() have
    all been moved into the global namespace to decouple them from the server
    class.

    Deprecation warning: These methods have been deprecated. The new
    json_get_timezone(), json_get_date_with_gmt(), json_get_avatar_url() and
    json_parse_date() methods should now be used instead.

    (props @rmccue, #185, #298)

As always, we’ve got a full list of all the changes and a longer changelog. Here’s who contributed to this release:

$ git shortlog 1.0...1.1 --summary
     8  Daniel Bachhuber
    12  DrewAPicture
     3  Eric Lewis
     2  JDGrimes
     9  K.Adam White
    54  Rachel Baker
   128  Ryan McCue
     4  Taylor Lovett
     1  jeremyfelt
     1  pkevan

Version 1.2

We’ve already started work on 1.2, and as always, we’re looking for help!

With version 1.2 and onwards, we’ll be tackling a bunch of extra testing for our endpoints, with the aim of eventually reaching >90% coverage. As always, we’ll also be adding new features and fixing bugs.

We’re also working on improving the new documentation site, and expect to see the majority of documentation migrated over there. Thanks to Sarah Gooding for helping out on the documentation side.

Core Integration

In case you missed it, the API is now slated for integration in WordPress 4.1. WP Tavern has a great writeup on the details.

As always, we look forward to seeing you at the team o2 and on GitHub. Now’s also a great time to remind you that you can get support for the plugin on WP.org, or by tweeting at me. Thanks to everyone who made this release great, and thanks to everyone using the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party!

#json-api, #rest-api

JSON REST API: Version 1.0

I’m incredibly excited to announce the availability of version 1.0 of the JSON REST API.

This version is a huge release and introduces a bunch of new features, such as user, revision and post metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. endpoints. It also introduces our long-term backwards compatibility policy, aligning with WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. backwards compatibility.

Here’s a selection of the new stuff:

  • Add user endpoints.

    Creating, reading, updating and deleting users and their data is now possible
    by using the /users endpoints. /users/me can be used to determine the
    current user, and returns a 401 status for non-logged in users.

    Note that the format of post authors has changed, as it is now an embedded
    User entity. This should not break backwards compatibility.

    Custom post types gain this ability automatically.

    (props @tobych, @rmccue, #20, #146)

  • Add post meta endpoints.

    Creating, reading, updating and deleting post meta is now possible by using
    the /posts/<id>/meta endpoints. Post meta is now correctly embedded into
    Post entities.

    Meta can be updated via the Post entity (e.g. PUT to /posts/<id>) or via
    the entity itself at /posts/<id>/meta/<mid>. Meta deletion must be done via
    a DELETE request to the latter.

    Only non-protected and non-serialized meta can be accessed or manipulated via
    the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.. This is not predicted to change in the future; clients wishing to
    access this data should consider alternative approaches.

    Custom post types do not currently gain this ability automatically.

    (props @attitude, @alisspers, @rachelbaker, @rmccue, @tlovett1, @tobych,
    @zedejose, #68, #168, #189, #207)

  • Add endpoint for deleting a single comment.

    Clients can now send a DELETE request to comment routes to delete
    the comment.

    Custom post types supporting comments will gain this ability automatically.

    (props @tlovett1, @rmccue, #178, #191)

  • Add endpoint for post revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision..

    Post revisions are now available at /posts/<id>/revisions, and are linked in
    the meta.links.version-history key of post entities.

    Custom post types supporting revisions will gain this ability automatically.

    (props @tlovett1, #193)

  • Respond to requests without depending on pretty permalink settings.

    For sites without pretty permalinks enabled, the API is now available from
    ?json_route=/. Clients should check for this via the autodiscovery methods
    (Link headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. or RSD).

    (props @rmccue, #69, #138)

  • Add register post type argument.

    Post types can now indicate their availability via the API using the
    show_in_json argument passed to register_post_type. This value defaults to
    the publicly_queryable argument (which itself defaults to the
    public argument).

    (props @iandunn, @rmccue, #145)

  • Remove basic authentication handler.

    This breaks backwards compatibility for clients using Basic
    authentication. Clients are encouraged to switch to using OAuth
    authentication
    . The Basic Authentication plugin can be
    installed for backwards compatibility and local development, however should
    not be used in production.

    (props @rmccue, #37, #152)

  • Require nonces for cookie-based authentication.

    This breaks backwards compatibility and requires any clients using cookie
    authentication to also send a nonce with the request. The built-in JavascriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/.
    API automatically handles this.

    (props @rmccue, #177, #180)

  • Clean up deprecated methods/functions.

    Functions and methods previously deprecated in 0.8/0.9 have now been removed.
    Future deprecations will take place in the same manner as WordPress core.

    This breaks backwards compatibility, however these were marked as
    deprecated in previous releases.

    (props @rmccue, #187)

  • Only expose meta on ‘edit’ context as a temporary workaround.

    Privacy concerns around exposing meta to all users necessitate this change.

    This breaks backwards compatibility as post meta data is no longer
    available to all users. Clients wishing to access this data should
    authenticate and use the edit context.

    (props @iandunn, @rmccue, #135)

  • Add json_ensure_response function to ensure either a
    WP_JSON_ResponseInterface or a WP_Error object is returned.

    When extending the API, the json_ensure_response function can be used to
    ensure that any raw data returned is wrapped with a WP_JSON_Response object.
    This allows using get_status/get_data easily, however WP_Error must
    still be checked via is_wp_error.

    (props @rmccue, #151, #154)

  • Use version option to check on init if rewrite rules should be flushed.

    Rewrite rules on multisitemultisite Used to describe a WordPress installation with a network of multiple blogs, grouped by sites. This installation type has shared users tables, and creates separate database tables for each blog (wp_posts becomes wp_0_posts). See also network, blog, site are now flushed via an init hook, rather than
    switching to each site on activation.

    (props @rachelbaker, #149)

As always, you can view all commits or the longer changelog.

For those interested, here’s the list of contributors to this release:

$ git shortlog --summary 0.9...
     1  Chris Marslender
     1  Eric Lanehart
     2  K.Adam White
     1  Kat Hagan
     2  Matth_eu
    41  Rachel Baker
   139  Ryan McCue
     5  Taylor Lovett
    10  Toby Champion

There’s a few really important things to note with this release.

Authentication Changes

Authentication has changed significantly in 1.0. If you’ve been using Basic authentication previously, you’ll now need to install the Basic authentication plugin. This pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party is designed for local development, as Basic authentication requires sending your plaintext credentials over the wire, which is unsafe for production.

Production users have two choices: built-in cookie authentication, or OAuth authentication. OAuth 1.0a is an authorization protocol that allows you to authorize clients to act on your behalf, and does not require giving your username and password to the client. It does, however, require a significantly more complicated authentication/authorization process, and clients are required to register on the site beforehand. We’re working on long-term solutions to this.

Plugins and themes can also use built-in cookie authentication. This is the normal WordPress login process, however requires a nonce for authentication to the site. This is automatically handled for you when using the built-in Javascript client.

Backwards Compatibility

From this release forwards, backwards compatibility will not be broken. This includes both the internal PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher API, as well as the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. we expose. New endpoints may be added, as well as new data, but responses will continue to be supersets of the current response.

The exception to this is for security concerns. As we continue development, we may need to change some endpoints for security issues, as we did with post meta for this cycle. These will be announced well before release where possible.

Please note also that this release has removed some previously deprecated methods and changed some internal implementation details. This only affects plugins or themes that extend the API.

Core Integration

We’re pushing hard for integration into 4.0 this cycle, and we need your help. Our core integration plan outlines the motivation behind the project and the specific plan for integrating it into core. We’re currently working on a comparison document for the API compared to the WP.com/Jetpack API and others, and will be publishing that soon.

We need your help to make it into 4.0. Developers, we’d love to know what you’ve built with the API, whether public or internal (even vague details help!), and we’d especially love to see things built with the API. We’re currently in danger of not making it in this cycle, so anything you can do to help us here would be fantastic.

As always, we’re also looking for help. The main API always needs help, and the other related projects do too. Both the WP-CLI client and Javascript client need help here.

You’re always welcome over at the team o2, and our next meeting will be at Tuesday, 00:00 UTC; we’d love to see you there. If not, see you soon for version 1.1!

#json-api, #rest-api

JSON REST API: Meeting Time Change

As announced on the team o2, the meeting time for the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. team is changing to Tuesday, 0:00 UTC, in #wordpress-dev.

This week we’re discussing OAuth (which, P.S., is now available for testing), 1.0 and custom post typeCustom Post Type WordPress can hold and display many different types of content. A single item of such a content is generally called a post, although post is also a specific post type. Custom Post Types gives your site the ability to have templated posts, to simplify the concept. data privacy.

Hopefully I’ll see you there!

#json-api, #rest-api