JSON REST API: Version 0.6 and The Future

We’ve finally come to the end of Summer of Code, so it’s time for the final GSoC release in this slightly late update (hey, would it be a post by me if not?). This release is mainly a stability release, so there are only minor changes:

  • Huge documentation update – Guides on getting started and extending the API are now available for your perusal
  • Add generic CPT class – Plugins are now encouraged to extend
    WP_JSON_CustomPostType and get free hooking for common actions. This
    removes most of the boilerplate that you needed to write for new CPT-based
    routes and endpoints (#380)
  • Use defined filter priorities for endpoint registration – It’s now easier to
    inject your own endpoints at a defined point
  • Update the schema – Now includes documentation on the Media entity, plus more
    (#264)
  • Add better taxonomy support – You can now query for taxonomies and terms directly. The routes here might seem strange (/posts/types/post/taxonomies/category for example), but the intention is to future-proof them as much as possible (#275)
  • Ensure the JSON URL is relative to the home URL (#375)
  • Check all date formats for If-Unmodified-Since (#378)
  • Register the correct URL for the JS library (#376)
  • Correct the usage of meta links (#379)
  • Add filters for post type and post status data (#380)
  • Separate parent post and parent comment relation (#330)

The big feature for this final week is a bunch of documentation changes. I’ve created guides and documents on how to use the API, intended for both beginners and experts. I’d especially like to highlight the Getting Started guide, as well as the Extending the API guide for plugin developers. The documentation in the past has been less than fantastic, so I’ve made a concerted effort towards it this week. In addition, I’ve also fixed up all the remaining bugs reported via Trac.

Now that GSoC’s over, what’s next? The aim with the project is to now move it from a solo project to a team one, and to that end, I’ve been working on assembling a fantastic team to work on the project with, with aim to integrate the API into core in the future. 3.8 time is fast approaching, so we’ve elected to aim for 3.9 as a more realistic target, although the advantage of the Feature as a Plugin method of development is that we’re not locked down here.

We’re held two meetings so far as a team, and I’ll announce a proper office hours time next week, but I’m also looking to try something new with the organisation of the team. More to come on that in the next team update, but in the meantime, you can check out the internal team discussion site. Anyone looking to get involved in the team is welcome to join as always, but I’d ask that only those serious about working on the project join, as there are a fair few people committed already.

Thanks to everyone, especially my mentors and Jen, for making this project a joy to work on so far. Here’s hoping we can keep the momentum as we push forward with the project.

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Version 0.5

This week, I finally have a new release for you! Version 0.5 is now available, with the following changes (and more!):

  • Add support for media – This has been a long time coming, and it’s finally at a point where I’m happy to push it out. Good luck. (#272)
  • Separate the post-related endpoints – Post-related endpoints are now located in the WP_JSON_Posts class. When implementing custom post type support, it’s recommended to subclass this.

    The various types are now also only registered via hooks, rather than directly in the server class, which should make it easier to override them as well (#348)

  • Add page support – This is a good base if you’re looking to create your own custom post type support (#271)
  • Switch from fields to context – Rather than passing in a list of fields that you want, you can now pass in a context (usually view or edit) (#328).
  • Always send headers via the server handler – Endpoints are now completely separate from the request, so the server class can now be used for non-HTTP/JSON handlers if needed (#293)
  • Use better error codes for disabled features (#338)
  • Send X-WP-Total and X-WP-TotalPages headers for information on post/pagination counts (#266)

As always, the full changes are available if you’re interested.

This week finally brings media into the fold. The process for uploading media is a little different to creating normal posts, so here’s how you do it.

First, upload the file via a POST request to /media. This can either be as a standard HTTP multipart body, with the name field set to file, or as a raw file in the body with the Content-Type header. (You can also optionally send a Content-MD5 header if you’d like the server to check the consistency of your file.) This will give you a 201 Created status, and point you to the new resource. You can now update that resource with the correct post data.

This multistep procedure is required to enable raw file uploads, and I’m not entirely pleased with it, but it’s the only way without requiring multipart requests. I’d love to have feedback on this system, as I think practical use will eventually reveal the correct method here.

So, it’s time to start winding up the Summer of Code portion of the project. There’s still one week left for the Summer of Code project, so you may still see a release next week, but most likely in the form of smaller updates, especially with documentation and testing. As I finish up, it’s time to look forward to the future of the project. The plan is to form a Feature as a Plugin team as we work towards core integration in future releases. People have already volunteered for the team back in the 3.8 scoping days, and I’ll be getting in contact with them shortly, but it’s not too late to nominate yourself for the team; let me know if you’re interested.

Thanks to everyone for testing and for your feedback. Stay beautiful.

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Coming Soon

It’s been a while since you’ve all heard from me, so I wanted to check in and assure you I am still alive. I’ve been plodding along behind the scenes with the JSON API and mostly getting design documents sorted.

The big feature I’m working on at the moment – media – has turned out to be tricker than I initially thought. While media is technically handled via custom post types, it’s a completely different kettle of fish to normal post types, so I’ve been working through the conceptual issues behind this to ensure that the API stays consistent both internally and externally. I really believe that it’s worth my time to sit down and consider these conceptual issues in depth rather than pumping out the feature as soon as possible. (The implementation of the media-related endpoints is working locally, but I don’t want to push this up while it’s still in flux.)

I still hold out hope to push through media, but will likely reduce the scope of the other tasks to compensate, due to the complexity of media and the time it has taken so far. I’d like to personally apologise for this, this is a result of improper scheduling for the second half of the project.

Personally, the past month or so has been pretty stressful as well, due to a number of other things going on in the background. Balancing this project with university work has become a big issue recently, so I’m working through this as best as I can. Ideally, my preferred option at this point would be to push this project out of the Summer of Code phase and into the open source team phase rather than continuing to work on the project in isolation.

Along those lines, revisions will be bumped from the Summer of Code phase completely. While they are part of core functionality, they’re a rather large task that is secondary in importance to media and also behind taxonomy handling. I’d love to see these in the plugin at some point, but that won’t be happening during the Summer of Code phase. What I would love is for someone to volunteer to develop this (in a separate plugin for now, thanks to GSoC’s restrictions) for integration back in as soon as possible, which would also help with validating the API’s usefulness.

So again, sorry and hopefully I’ll have something better to show you next week. Thanks for your patience.

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Version 0.4

After a week’s hiatus thanks to WCSF and the midsemester review, I’m back with a new release of WP API! Version 0.4 is now available with the following changes:

  • Add Backbone-based models and collections – These are available to your code by declaring a dependency on wp-api (#270)
  • Check json_route before using it (#336)
  • Conditionally load classes (#337)
  • Add additional test helper plugin – Provides code coverage as needed to the API client tests. Currently unused. (#269)
  • Move json_url() and get_json_url() to plugin.php – This allows using both outside of the API itself (#343)
  • getPost(0) now returns an error rather than the latest post (#344)

As always, the full changes are available if you’re interested.

This release brings the first version of the Javascript API, based on Backbone models and collections. To use it, simply declare `wp-api` as a dependency to your script, then start using the `wp.api` classes. Here’s a quick example of how to use it:

var posts = new wp.api.collections.Posts();
posts.fetch({
	success: function (posts) {
		var post = posts.at(0);
		var title = post.get('title');
		post.set('title', title + ' (Updated!)');
		post.save();
	}
});

These are intended purely as building blocks for your own software. I had been looking at rewriting P2 partially to use these, however it appears that would require gutting P2 and basically starting from scratch, due to P2’s architecture. I’d love to see what you can do with this though, and bonus points if you can get a API-ified P2!

The coming week will introduce some specialised page handling as an example of how to enable custom post type support, plus the beginning of the media/attachment-related APIs. These will probably be a fair bit of work, so it’s possible only basic functionality will land next week.

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Testing Updates

This week is a little different to the previous ones, as it’s a primarily testing focussed week. I’m skipping the release for this week in favour of just an update post.

I noticed fairly late this week that the aforementioned client library wasn’t actually public and available to you guys, so I’ve now fixed that up. Given the lack of comments, I’d hazard a guess this also means no one tried using it. 🙂

Previous Week

Over the past week, I’ve been looking at a variety of testing-related items, including code coverage and schema validation. I’ve finally sorted the schema validation out with the help of a JSON Schema validation library, after much messing around. The unit tests now integrate this schema validation with the normal testing suite, which should ensure that the API is conformant to the specification as well as fully functional. I do suspect that the schema is slightly outdated and is missing a few items, so I’ll be ensuring the documentation and schema are consistent in the next week.

Another area of the testing that I’ve been looking at is working with code coverage. After exploring the inner workings of PHPUnit’s code coverage, I think I’ve worked out a solution using a test helper plugin. This gathers the statistics on the server, then serialises them to send them back to the client. This still needs the PHPUnit end to connect to, which should be simple once I work out how to override the PHP_CodeCoverage object. (Help here would be appreciated if anyone has familiarity with PHPUnit.)

I’ve also started work on the Backbone client, however that’s currently not in a state to release. The plan here is to create a generic set of Backbone-based classes that can be used as a library, with the proof-of-concept itself being a theme based on P2. I’ve started work on the Backbone side of things, however I’ve not worked on much here with the theme side. I suspect that P2’s Javascript will be discarded in large part, but hopefully I can avoid that as much as possible.

Next week, you can look forward to more testing updates and the Backbone client. I’ll also be pushing out a new version with some bug fixes (thanks to Mike Schinkel for reporting these) as well as the usual slate of updates.

Documentation

I’ve also discovered that my documentation thus far has been a little lacking, so here’s a bit of an intro to using the API.

After downloading and activating the plugin, your site will now capable of serving up data from its API. The API is accessible at /wp-json.php from your site (if you’re not sure where this is, copy your admin URL and replace wp-admin with wp-json.php), with the specific route added after this. For example, the index is available at /wp-json.php/ and a list of your posts is available at /wp-json.php/posts.

If you’d just like to take a look around, you can view this in your browser and navigate via the links in the JSON. I personally use JSON View for this in Firefox, which converts the URLs in the data into proper hyperlinks as well as making it easier to view the data. The API uses a concept called HATEOAS, which basically means that all the possible data is discoverable via these links.

Some endpoints will add extra data if you’re logged in with the correct permissions. You can log in via the normal WP login page if you’re just viewing this in your browser (using cookie-based authentication), but if you’re accessing the API programatically, the API also has built-in support for HTTP Basic authentication. (OAuth support is not planned for the core of the plugin, but my OAuth provider plugin might be a good start for anyone who wants to write this as a plugin.)

I’m always open to questions regarding how to use this, so let me know here or on Twitter if there’s anything I can do to help you out.

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Version 0.3

Time for another release of the JSON REST API! We’re closing in on midsemester evaluation, so it’s time to get our test on and ensure that everything’s working well. Only small updates to the API itself this week, but also the first introduction of the comment API:

  • Add initial comment endpoints to get comments for a post, and get a single comment (#320)
  • Return a Post entity when updating a post, rather than wrapping it with useless text (#329)
  • Allow filtering the output as well as input. You can now use the json_dispatch_args filter for input as well as the json_serve_request filter for output to serve up alternative formats (e.g. MsgPack, XML (if you’re insane))
  • Include a profile link in the index, to indicate the JSON Schema that the API conforms to. In the future, this will be versioned.

The big part of this week is the introduction of the testing suite and reference client. These are now both available on GitHub and I’d love for you to run them and test out the API. There’s currently one known bug that causes one of the tests to fail, which I’ll be handling in 0.4 due to the large scope of the issue.

I’m planning to continue work on the test suite this week, due to the large scope of the testing suite. Ideally, I’d like to get up around the 90% mark for code coverage (which I’ve done previously on my Requests library), however I’m not sure as to how coverage can be measured in this case, so I’ll also be investigating this. The coming week will also involve creating a Backbone-based sample theme as a proof-of-concept of real world use. In addition, coming up I’ll be consulting with Max Cutler from the mobile team and ensuring that any possible issues are sorted out at this early stage.

As always, I appreciate all and any feedback on the project. This week and the next will be boring for you guys in terms of new features as we approach midsemester review, so I’d love you to get out there and break the API. If you can find problems with it, please file them and let me know so I can work them into the testing suite.

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Version 0.2

It’s that time of week again folks (and actually on-time this time)! Version 0.2 of the JSON REST API includes my changes from the past week, including:

  • Allow all public query vars to be passed to WP Query – Some private query vars
    can also be passed in, and all can if the user has edit_posts
    permissions (#311)
  • Pagination can now be handled by using the page argument without messing
    with WP Query syntax (#266)
  • The index now generates links for non-variable routes (#268)
  • Editing a post now supports the If-Unmodified-Since header. Pass this in to
    avoid conflicting edits (#294)
  • Post types and post statuses now have endpoints to access their data (#268)

You can also view all changes if that’s your style.

As you can see, it’s a bit of a quiet week in terms of new features, but with the exception of revisions, attachments, and comments, the post API is now considered feature complete. Implementers can now expect that there won’t be any major changes to the post API.

Since you’ve all been so great, I’ve thrown in a bonus for this week. I’ve created a small proof-of-concept of extending the API for a multisite installation, which is now available via GitHub. Keep in mind that this is just a proof-of-concept, so it doesn’t have too much implemented. (This is also not covered under the scope of the GSOC project, so pull requests are most welcome!)

In the next couple of weeks, development will be focused around creating tests and example implementations. While this is occurring, I plan to also implement commenting, which was not in the original scope, but is still fairly important.

As always, feedback is welcome!

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API: Version 0.1

It’s time for the first weekly release of the API plugin for everyone to sink their teeth into. This post comes to you a few days late, as I’ve unfortunately been a bit sick during the week.

You can either install it from GitHub or from WordPress.org.

Here’s what’s changed since the pre-GSoC version:

  • Enable the code to be used via the plugin architecture (now uses rewrite rules if running in this mode)
  • Design documents are now functionally complete for the current codebase (#264)
  • Add basic writing support (#265)
  • Filter fields by default – Unfiltered results are available via their corresponding *_raw key, which is only available to users with edit_posts (#290)
  • Use correct timezones for manual offsets (GMT+10, e.g.) (#279)
  • Allow permanently deleting posts (#292)

You can also check out the full changelog if you’re interested.

Astute observers amongst you will notice that the main collection parts (#266, #267) are missing from this release; I’m slightly behind on the release process due to just finishing up final exams and illness. Don’t worry though, I preplanned for this by allowing myself a buffer week before the midsemester deadline.

Feedback is very welcome!

#gsoc, #gsoc2013, #json-api, #rest-api

JSON REST API

Hi everybody! Some of you may know me from various patches or WP-related endeavours: I’m Ryan McCue, developer of the SimplePie library used in core, along with a HTTP library called Requests, and long-time core hacker. I’ve been working with WordPress for quite a while now, both on open source and professional work with clients. What you may not know is that I’m also studying Electrical Engineering and Maths at UQ in Australia, and I’m here to give you a heads up on my Summer of Code project over the coming months.

For those who missed the discussion on wp-hackers about my proposal, I’m working on a JSON-based REST API for core. I started on this with an initial proof-of-concept back at the end of last year, and I’m now working on expanding this out into a larger project. The code is being written as a plugin for testing, with the goal of having it integrated into core in some form post-GSOC.

I’m planning on following a release strategy similar to MP6, with a weekly release along with the updates included in the release. At the moment, I’m working on completing the basic reading and writing of post data having just completed the major design documents, and I’m hoping to get the first weekly release out next week. I have a more detailed timeline which you can check out in my announcement post on my blog.

(You’ll notice I’m currently about a week behind on my schedule, which I suspected may happen, as I’m in the midst of my final exams here. I’ve allocated an extra week just before the midsemester review for catching up if I don’t do so before then.)

As it is, the plugin is in a usable format, and you can grab it from either GitHub or Subversion. I’d also recommend checking out the GSOC Trac issues if you’d like to keep track of the status. I’d love to have your feedback (especially on the design documents) as I move forward.

Cheers, and I look forward to working with you all in the coming months!

#gsoc, #gsoc2013, #json-api, #rest-api

GSoC 2013

It’s time for us to apply for GSoC 2013 if we want to participate. I can take care of the application, but will need everyone’s help in producing the project ideas and mentor sections.

“Why didn’t we get accepted last year?”

We didn’t get accepted last year because our Ideas page wasn’t as good as it had been in past years, and they wanted to make room for some new orgs, so we got cut. Our ideas page wasn’t great because we were planning to embed students in regular cycle feature teams, but we didn’t write up all the potential features this might have included. This year we need to write out all the possible features.

PROJECT IDEAS:

The more project ideas the better, so things that aren’t targeted for the next cycle but that we might want to do as a plugin to get some traction for inclusion in a later release would be great projects, too. Project ideas can include core features, plugins, themes, unit tests, etc. This includes work on .org site things like the plugin used to power WordCamp sites. The projects must be code, though, not design, documentation, etc. List your project ideas in a comment on this post. Please give it a title, and a short one-paragraph description. If there’s a ticket or relevant thread somewhere, link to it.

POTENTIAL MENTORS:

Want to volunteer to be a mentor? You need to be good enough with wp coding that you currently don’t need oversight to generally get it right (though obviously we all can get better with more eyes on code), and have enough time to be available to your student for feedback and code review (at least once per week, though more often is better). The time commitment can be anywhere from one hour to ten hours per week, depending on how far along your student’s project is, and what there is for you to review (an hour or two a week is typical). You get a GSoC tshirt at the end. Want to be a mentor? Leave a comment on this post with a short/one-paragraph little bio about yourself, what kinds of projects you are interested in/qualified to mentor, and links to your website, wordpress.org profile, and twitter account.

LEAD DEVELOPERS:

I know you are right smack dab in trying to get this release out, which is why I’ll take care of the application and associated materials. That said, I’m a bit out of the loop with core plans, so coming up with the potential projects list on my own would mean they might not mesh with the actual goals/plans of core team very well. I’ll be looking to you guys to give me a sense of priority/possible plans for the next dev cycle, and to tell me who is qualified to mentor a project from the final list of volunteers. Will ping you offline for this.

Not sure how to write up a project idea or your short mentor bio? Take a look at the ideas page from 2011 for a guide.

Note 1: This year BuddyPress will be applying as a standalone organization with WP as the vouching organization. I’ll be helping Paul Gibbs get their application going.

Note 2: I haven’t announced it yet since it just happened, but we’ve been accepted as a mentoring organization for the Gnome Outreach Program for Women for the summer session. Unlike GSoC, where Google foots the bill, we’ll have to raise the money to cover our interns for the Gnome program, so I’ll announce it along with a fundraising drive. The Gnome program is a little broader than GSoC, and does allow non-code projects, so things like design, ux, documentation, translation, community management, etc would all be possibilities there.

#gsoc, #gsoc2013