Show Me Your WP REST API v2 Apps

WordPress 4.4 development hit the ground running last week, only a few hours after the launch of 4.3. We’re already close to 100 commits, and digging through the 385 responses on my “what’s on your wishlist” post. One feature that many of you want is the WP REST API (heard of it?). Lots of work has gone into it, and some people are already using a flavor of it in production – two that I know of:

  • Wired uses the REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. for communication between some of their internal systems
  • the New York Times for our Live Coverage platform (example – Republican Debate: Analysis and Highlights)

Both use version 1.*. I am working on an upgrade path for the NYT to version 2.

The point of this post is to solicit feedback from the general community:

  • What have you built using v2 of the REST API?
  • Are you running the project in production? If so, please post a link 🙂
  • Have you upgraded from v1 to v2? If so, how was it?
  • If you believe in the project, would like to see it in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., and haven’t built anything with it: what’s stopping you?

Let’s take the excitement everyone has for the feature and start stress-testing it. Build something. Anything! And then report your findings back here.

#4-4, #rest-api

WordPress 4.4: What’s on your wishlist?

4.4 has unofficially kicked off, now that 4.3 is out the door. As with any release, we want to garden TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress., squash bugs, add new tools for dev, and wow our users.

I have spoken to many of my fellow core contributorsCore Contributors Core contributors are those who have worked on a release of WordPress, by creating the functions or finding and patching bugs. These contributions are done through Trac. https://core.trac.wordpress.org. about their own wish lists. Now, it’s your turn:

  • What do you want to see happen in 4.4?
  • What are pain points for users?
  • What features can we add or iterate upon to empower our user base?

It can be anything: big or small.

As most of you know, I am leading WordPress 4.4

If we’ve never met, hello! You can learn a lot about me here:

 

#4-4, #wishlist

Trac Tickets – the band’s taking requests

At the beginning of the past few releases or so, we’ve put a call out for community priority tickets. There are over 3000 tickets in TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress.. If there’s a ticketticket Created for both bug reports and feature development on the bug tracker. you feel is neglected or should have light shined upon it: leave a comment here with a link so we can triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. it. Help us help you.

The Case for JS Modules

I originally posted some of this content here: Split javascript files in media into modules

The patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing. on that ticketticket Created for both bug reports and feature development on the bug tracker. breaks up the Backbone classes in media-models.js, media-views.js, media-audiovideo.js, and media-grid.js into modules and injects them via Browserify on build/watch into a built file. Let’s start at the beginning.

Brain overload

Files that are 1000s of lines long are hard to consume. We try to alleviate this by adding copious amounts of docs. Still, it’s a lot to look at. Ideally, we would break our files into smaller modules and then somehow join them together in a build process.

It is common practice to serve (sometimes very large) minified files for JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors. and CSSCSS Cascading Style Sheets. that have concatenated many smaller files together and uglify’d (minified/obfuscated) them. It is no longer common or best practice to develop with huge files. We can learn a lot from emerging front end development trends, especially those from the Node/NPM community. In some cases, we can even share code.

We’ll use Media as the main culprit, but this could apply to any “manifest” – a term I use to describe files that contain the entire public APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. for a feature. Something like media-views.js, it might be nice to bounce from view to view in the same file, provided you know exactly what you are looking at, what depends on what, etc.

I have found, it is completely overwhelming for almost everyone. It would be great if each discreet piece could be viewed in isolation with its dependencies clearly stated.

There are many ways to accomplish the splitting of large files. I want to focus on 2 of the most common.

Vocabulary

Backbone is one of a growing number of MV* frameworks for JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/.. A large majority of the code related to media either belongs to a handful of Models or to the increasingly large library of Views and View Templates.

Views are the building blocks for the presentation of Media (you know, “the Media Modal” or 4.0’s “Media Grid”).

The main canvas on which these Views are stitched together are called Frames, which are themselves Views – tilting our use of Backbone more towards MVPMinimum Viable Product "A minimum viable product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development." - WikiPedia, P standing for Presenter.

We have Controllers, which are called States, but they belong to a Frame (Presenter! also a View!), so anyways…. for now….

When we create new UIs, we are more than likely adding new Views/Templates, or updating existing Views.

If we wanted to move from one large file to many files that each contain a class, we would create Modules.

Grunt is a task runner. We use Grunt to build our src directory into our build directory.

Require

Require is a great tool for converting AMD modules into built files. Require leans on Dependency Injection in its syntax:

define([
    'models/taco',
    'models/burrito',
    'controllers/meal'
], function (Taco, Burrito, Meal) {
    var Dinner = Meal.extend({
        // taco-related code
    });
    return Dinner;
});

This syntax works great, unless you have way more dependencies. Refactoring code could unwind a module that has a lot of dependencies, but if you are just trying to convert legacy classes into a module, Require starts to get a little weird. Concrete example: Frames have a TON of dependencies.

Require becomes a Grunt task to make one big file by recursing the dependency tree in an initial manifest. Require, by default, loads JS asynchronously, which can cause race conditions with plugins or themes that expect code to be registered on $(document).ready() or window.onload.

Require works even if you don’t build via Grunt.

Browserify

Browserify is a tool that allows you to use Node-style modules and run them in a browser without changing from the Node syntax. Browserify requires a build for this to work.

Using our example from above, this is the syntax for Browserify:

var Taco = require( './models/taco.js' ),
    Burrito = require( './models/burrito.js' ),
    Meal = require( './controllers/meal.js' ),
    Dinner;

Dinner = Meal.extend({
    // taco-related code
});

module.exports = Dinner;

Browserify leans more towards the Service Locator pattern.

Browserify scans the abstract syntax tree (AST) of your JS code to compile dependencies. Your modules themselves get wrapped in their own scope like so:

(function (require, module, exports) { 
    .....YOUR_MODULE..... 
})

After spending a lot of time messing around with both: I think we should use Browserify.

Converting “Legacy” Code

The media JS code is some of the most “modern” code in WordPress, but it still clunkily lives in huge files. To convert the code into modules, we need to make a lot of individual files (one for each Backbone class).

We also need to make sure we maintain the existing wp.media namespaces for backwards compatibility. We don’t want any existing functionality to change, we just want to build the files differently.

Even though the code is defined differently, wrapped in a new scope, and looks different when “built”, we can still maintain our current API design: what is publicly accessible now will remain publicly accessible.

In the patch

Disclaimer: this patch is for experimentation only. It will go stale probably before this post is published. It works, but it is only a playground for now. If this moves forward, it will be a laborious Subversion process to create a bunch of new files.

I have added a folder to wp-includes/js, media, that contains the modules and the built manifests. My patch adjusts script-loader.php to use these new paths.

media contains the following files/folders:

controllers/
models/
routers/
utils/
views/ (with another round of subfolders)
audio-video.manifest.js
grid.manifest.js
models.manifest.js
views.manifest.js

The build pipeline

If you are following along with that patch and want to see this in action, run in the project root:

npm install

Afterwards, run:

grunt watch

*.manifest.js files get built into *.js files when you change a file in media/*, provided you are running the grunt watch task. The watcher will automatically call browserify:media and uglify:media when those files change. This allows you to run your site from src or build, and you will still get Browserify’d files. SCRIPT_DEBUG will either run *.js or *.min.js, just like any other minified JS in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress..

This is a proposal

I would like to hear feedback from the overall community and certainly from our fair share of JS-trained ninjas. A common reason to *not* do something like this is the barrier to entry for new developers. I would argue in this case that the code becomes MORE readable and understandable. I was shocked myself to see how much simpler it was to absorb one piece at a time once the code was laid out in modules.

#backbone, #js, #media

4.1: Call for Tickets

What tickets would you like us to look at in 4.1? (Extra points if it already has a patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing..)

Since the release is officially underway, now is a great time for us to prioritize work and make sure our pet projects/tickets get some airtime. The priorities of the community matter. Let us know below what you would like to see in WordPress 4.1.

#4-1

A more powerful ORDER BY in WordPress 4.0

orderby is the argument passed to WP_Query to tell it what column to sort on when it is creating the ORDER BY clause for its generated SQL. The default value for orderby is post_date.

The default sort order for a column in MySQLMySQL MySQL is a relational database management system. A database is a structured collection of data where content, configuration and other options are stored. https://www.mysql.com/. is ASC (ascending), with smallest values first. For the reverse, DESC is used. You can sort on multiple columns, and each column can have its own sort order.

The default value for the order argument inside WP_Query is DESC. ~23% of the internet automatically queries posts in reverse chronological order because of this. order can only be one of 2 values: DESC or ASC.

orderby accepts a string, representing a column on which to sort:

$q = new WP_Query( array( 'orderby' => 'post_title' ) );

// or an alias
$q = new WP_Query( array( 'orderby' => 'title' ) );

Both will produce an ORDER BY clause like:

ORDER BY post_title DESC

orderby will also parse a space-delimited set of columns:

$q = new WP_Query( array( 'orderby' => 'title author' ) );

Prior to 4.0, there was a problem: the value for order would only be applied to the last value that you passed in that space-delimited list, producing an ORDER BY clause like:

ORDER BY post_title, post_author DESC

Remember that the default sort order for a column in MySQL is ASC, so queries like that can get weird real fast and produce unexpected/unpredictable results. If no value is passed for order for a column in the generated SQL, the column will be sorted in ASC order. This was not so clear to all developers. #26042 was a joy to debug.

In 4.0, when you pass a space-delimited set of values, your sole value for order will be applied to all of your values that are parsed for orderby. This was fixed in [28541].

So that’s pretty good, but it doesn’t allow you granular control over the sort order for each column. The syntax doesn’t allow itself much room for extending.

Enter [29027].

In 4.0, you can now pass an array to WP_Query as the value for orderby. The syntax looks like:

$q = new WP_Query( array( 'orderby' => array( 'title' => 'DESC', 'menu_order' => 'ASC' ) ) );

This allows you to control the generation of the ORDER BY clause with more specificity:

ORDER BY post_title DESC, menu_order ASC

Pre-4.0, you would have had to use some gnarly filters on the SQL statement or a specific clause. No bueno.

To see the internals, check out the new protected methods in WP_Query: ->parse_order() and ->parse_orderby.

Happy WP_Querying!

#4-0, #dev-notes, #query

like_escape() is Deprecated in WordPress 4.0

@miqrogroove has written a blog post on his personal blogblog (versus network, site) explaining why like_escape() has been deprecated in WordPress 4.0. It has been reposted below.

PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party authors and website developers who work with WordPress database queries should notice an important change coming in WordPress 4.0.

The function like_escape() is no longer used in WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. code. It is still available as a deprecated function, so it still works in any existing plugins that rely on it. However, a new and different function is available that should be used in all new code.

Deprecated means that anyone using code that calls like_escape() with WP_DEBUG enabled will see an error message. If WP_DEBUG_LOG is also enabled, the error message will appear in the /wp-content/debug.log file.

Let’s look at an example of core code where I removed like_escape() and implemented the new function $wpdb->esc_like().

3.9 Old Style

$search_orderby_s = like_escape( esc_sql( $q['s'] ) );
$search_orderby .= "WHEN $wpdb->posts.post_title LIKE '%{$search_orderby_s}%' THEN 1 ";

What did this do? It was an old snippet from /wp-includes/query.php that set up a search for post titles. The input $q['s'] was escaped using two functions before it was added to the post_title LIKE expression. Now let’s see how I replaced that snippet in the next version.

4.0 New Style

$like = '%' . $wpdb->esc_like( $q['s'] ) . '%';
$search_orderby .= $wpdb->prepare( "WHEN $wpdb->posts.post_title LIKE %s THEN 1 ", $like );

There are two important differences to notice.

  • I changed the like_escape() call to $wpdb->esc_like().
  • I changed the esc_sql() call to $wpdb->prepare().

The second change is important because esc_sql() is not secure if it is called before, or inside, the call to the new function $wpdb->esc_like(). By relying on the preferred style of the function prepare(), I can easily see that each instance of $wpdb->esc_like() will run first instead of last.

4.0 Alternative Style

Here is something that still works, but I avoided using it. Notice the old query is unchanged. It is critically important to call the two escaping functions in the correct order when using $wpdb->esc_like().

$search_orderby_s = esc_sql( $wpdb->esc_like( $q['s'] ) ); // This is the correct order.
$search_orderby .= "WHEN $wpdb->posts.post_title LIKE '%{$search_orderby_s}%' THEN 1 ";

How Should I Get My Code Ready for 4.0?

The nice thing about deprecated functions is that you can still use them and they don’t change. Your existing code should work fine.

When you write new code, remember that using $wpdb->esc_like() is not compatible with WordPress 3.9. This should be avoided if you need compatibility with old versions. When you are ready to adopt 4.0 as your minimum version, consider using the new function.

If you have a specific need for the new function in old versions of WordPress, I suggest copying the new function into your plugin under a different name. This would be the simplest solution, but rarely necessary.

Why Did like_escape() Change to $wpdb->esc_like()?

There were several problems with the old function that could not be fixed.

  • Documentation said the function’s output was safe to use in SQL queries. That was not correct.
  • The function’s output was not fully compatible with LIKE expressions in MySQLMySQL MySQL is a relational database management system. A database is a structured collection of data where content, configuration and other options are stored. https://www.mysql.com/..
  • The function had been used many different ways in core code, some of which were incompatible with the desired output.
  • Changing the old function instead of creating a new one would have caused many security problems in plugins.
  • The function was related to $wpdb because of its MySQL syntax, which does not work on other databases.

Is There a Security Problem with like_escape()?

The old function like_escape() was not intended to be used in any security sensitive context. There are no security problems when it is used correctly.

With that said, I am concerned that plugin authors frequently confused the old function like_escape() with esc_sql(), which was used for security. The documentation for like_escape() was misleading and very confusing about this point.

Just remember, like_escape() does not provide any security for your database!

So What Does $wpdb->esc_like() Do Anyway?

Whenever user input or other raw data are copied into a WordPress query, the data must be escaped using $wpdb->prepare() or esc_sql(). This prevents certain characters, such as quotes, from getting confused with SQL commands.

In a LIKE expression, there are additional special characters that are used as wildcards to search for partial matches in the database. Those wildcard characters have to be escaped from the user input so that they are not confused with the wildcards added by the programmer.

Before adding user input to this type of search query, $wpdb->esc_like() should be called for compatibility, and then $wpdb->prepare() must be called for security, in that order.

How to Use $wpdb in Functions

It is very rare to use $wpdb->esc_like() without also running a query. But just in case you want to …

function my_search( $input ) {
    global $wpdb;
    $escaped = $wpdb->esc_like( $input );
    ...
}

… remember to reference $wpdb as a global variable.

#4-0, #database, #dev-notes, #query, #security

Community Priorities

Triaging tickets takes a lot of time and is often thankless work. Testing patches and making sure they are committable is equally as time-consuming. Being a gatekeeper to those tickets going into coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. is also tough – sure, you can commit anything, but you are also the first person responsible when it breaks or there weren’t unit tests in place (when there should have been).

As a community, we often have 2 major goals:
1) Fix everything
2) Make everything better

Since we are still in the midst of defining what 4.0 will ultimately be, I would like to ensure that the community once again has a voice as far as “priority” tickets go, outside of dev chat.

If there is an important ticketticket Created for both bug reports and feature development on the bug tracker. that you feel has been neglected, comment below. Seriously. There is always frustration from those who say “my ticket never gets looked at!” – for anyone feeling especially disillusioned, remind us publicly to give you some feedback. This does not mean that every ticket deserves to ultimately go in right away, but let’s talk about it.

Speaking for myself, I often sit down at the computer and say “my goal today is to commit 100 tickets!” – I usually end up committing 8, over the course of 5 hours. And most of those tickets haunt me for weeks. Everyone is held to a high standard and expected to follow through when they make a decision for everyone else.

We ALL want the same thing. Let’s work together to get there!

Multisite: let’s arrange office hours

I am envisioning lots of updates to Multisitemultisite Used to describe a WordPress installation with a network of multiple blogs, grouped by sites. This installation type has shared users tables, and creates separate database tables for each blog (wp_posts becomes wp_0_posts). See also network, blog, site for 4.0. @johnbillion and @jeremyfelt have been taking the lead. We should meet (at least) weekly to check in, plan work, and track progress. Not too formal, just to keep it in motion.

Questions:
When is a good time to meet?
Who would like to be involved?

Leave a comment / suggestion.

There is no master plan. There is a make/core post: https://make.wordpress.org/core/2013/10/06/potential-roadmap-for-multisite/

Let’s discuss.

#multisite

Audio/Video 2.0 Update – Media Modal

The latest major updates to Audio/Video before BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. were related to editing shortcodes in the medial modal. TinyMCE placeholders for audio and video have been added like so:

00-placeholders

When you click the placeholder, the media modal will now be launched. And when you get there, there are some new ways to manage HTML5 playback without writing anything by hand.

01-audio-details

Add Multiple Sources

MediaElement.js provides cross-browser support for many extensions by providing Flash and Silverlight files that will bridge the browser support gap when necessary. Ideally, as many browsers as possible would be using native HTML5 playback. To ensure this, you need to specify multiple versions of your files. Since 3.6, the audio and video shortcodes have supported this by allowing you specify multiple extensions as attributes: mp3="yolo.mp3" ogg="yolo.ogg" wma="yolo.wma", etc.

A quick and easy way to encode your files is to use FireFogg (works in Firefox only). Here are some instructions for audio and video files:

HTML5 Audio

  • Click “Make web video”
  • Select a File (your iTunes files on a Mac are in: ~/Music/iTunes/iTunes Media/Music – Pick a tune!)
  • Click “Advanced Options”
  • Uncheck Video, make sure Audio is checked
  • Format: Ogg (Theora/Vorbis)
  • Set quality to 10.0
  • Optionally add metadata
  • Click “Encode” – make sure to change the extension in the file prompt from “.ogv” to “.ogg”

HTML5 Video

  • Click “Advanced Options”
  • Make sure Audio and Video are both checked
  • Format: choose Ogg (Theora/Vorbis) or WebM (VP8/Vorbis)
  • Optionally add metadata
  • Click “Encode”
  • (Repeat these steps for the format you didn’t select this time)

There is now a workflow to make adding these extra sources to shortcodes easy:

02-audio-add-ogg

Multiple sources are specified now. Make sure to click blue “Update” button to save your shortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. back to the editor.

03-audio-sources

Video Workflow

Here is a video workflow, assuming your shortcode has no attributes and you click it:

04-video-empty

Add each video source:

05-video-add-source

Optionally add a poster image:

06-video-sources

Subtitles

If you’re feeling CRAZY, add some Subtitles! Here’s a post about them. They’re pretty cool. Probably make sure your web server is serving .vtt files as text/vtt.

07-webvtt

Add your subtitles using our easy workflow:

08-video-add-subtitles

When you’ve added them, MediaElement knows to support them out of the box:

09-video-click-subtitles

Boom. Subtitles while your video plays:

10-video-show-subtitles

When you add your subtitles, if will show you a list of “track” elements you have added, you will still need to set your language manually – all will default to English. The idea is that you can add a track element per language. Tracks get stored as the body of your video shortcode.

tracks

Testing + Tweaks

Now comes the time for testing and refining, while fixing esoteric bugs. PLEASE HELP! Put your UIUI User interface + UXUX User experience hats on, if you wear that kind of hat.

#audio, #media, #mediaelement, #video