Audio / Video support in Core

Post Formats are a big feature in WordPress 3.6. What you may not know is: there is now native support for Audio and Video in core! There has been great support for embeds by way of WP_Embed and oEmbed providers for a while, but, if you wanted to play an MP3 from your Media Library, you had to install a plugin. Supporting audio and video in core gives bands, podcasters, vloggers, et al the ability to easily and beautifully expresses themselves through sounds and moving pictures without using an external service.

How does this work?

At the core of the experience is the fantastic library, MediaElement.js. MediaElement is the facade layer that gives us maximum file support and cross-browser compatibility. While some libraries require a Flash-only solution to make your media work cross-environment, MediaElement lets you use HTML5 audio / video tags in every browser, and, only when necessary, will use a Flash or Silverlight plugin in the background to make incompatible media work. Translation, things like this: <audio> tag works in old IE, Windows Media files work in Chrome.

MediaElement uses the same HTML markup, regardless of playback implementation, and you can use CSS to skin the players.

Shortcodes

MediaElement’s great, but we don’t want to be locked in to one external library forever. Instead of using MediaElement-specific markup everywhere, we expose audio and video markup through shortcodes: [audio] and [video].

For the following scenarios:

  • I have an old post that has a video in the Media Library attached to it, and I want to use the new shortcode: [video]
  • I have the URL for a video, from the Media Library or external, that I want to play:
    [video src="video-source.mp4"]
  • I have a source URL and fallbacks for other HTML5-supported filetypes:
    [video width="600" height="480" mp4="source.mp4" ogv="source.ogv" webm="source.webm"]

Same goes for audio:

  • I have an old post that has an audio file in the Media Library attached to it, and I want to use the new shortcode: [audio]
  • I have the URL for an MP3, from the Media Library or external, that I want to play: [audio src="audio-source.mp3"]
  • I have a source URL and fallbacks for other HTML5-supported filetypes:
    [audio mp3="source.mp3" ogg="source.ogg" wav="source.wav"]

Shortcodes focus on the “what” and abstract the “how.” If you want to use a library that is not MediaElement, you can! Just look at what to filter: here

Embeds

There are also new embed handlers for audio and video. Using them is easy as dropping a media link on a line by itself in the editor:

http://my.mp3s.com/cool/songs/coolest.mp3

I like this song because it is really cool!

Works for both audio and video with URLs matching the allowed (and filterable) list of extensions (see: here and here)

Admin

Using the new post formats UI, it is even easier to get directly at the audio and video in your Media Library. When selecting, the media modal opens to your library, filtered by media type.

Metadata

In previous versions of WP, you could upload audio and video, but we were not generating metadata like we do for images. In 3.6, using the getID3 library, we are able to extract data from audio and video like cover art, song length, artist, album, song title, genre, codec, etc. It’s pretty great. We will soon be exposing more of this data in the admin as well, along with inline previews on the Edit Media page:

Themers

Themers can get in on the action, too, using structured-post-formats in their theme (Twenty Thirteen is a great place to look). The admin gives users flexibility when associating media with a post. the_post_format_audio() and the_post_format_video() will automagically retrieve and output your media in the front end.

#audio, #media, #mediaelement, #post-formats, #shortcodes, #video

Post Formats UI Update, 3/14

As noted in The Road to 3.6 Beta 1, we’ve got quite a bit going on for post formats. Many of the tickets are in need of testing (including unit tests) and then a commit. As always, there are a few different fronts: UI/administration, data, and parsing. Here’s where we are with each, and what needs to get done. There’s a large variety of tasks here, and we are seeking contributors to help 🙂

Continue reading

#3-6, #post-formats

Post Formats UI Update, 2/21

First run commit, mostly for soaking of functionality, went into trunk on Monday. This includes fallback output for theme compat and fields in the admin. The admin UI does not represent anything final – especially of note is the tabs, which are essentially the way they are in order to reuse CSS and avoid adding temporary code to core.

Continue reading

#3-6, #post-formats

Post Formats UI Update, 2/4

Not a whole lot to discuss today – working away at patches on #19570 (edit form UI) and #23347 (theme compat). Could use a patch on #15490 (retrieval of oEmbed data and previewing, which I suspect will become relevant to the edit form UI) and testing of #23282 (audio/video shortcode and player) and #16047 (list table column).

#3-6, #post-formats

Post Formats UI Update, 1/31

Had really good conversations in Wednesday’s dev chat and continued in today’s office hours. Seems like we’ve got a really good direction. All the UI will be available all the time and theme support for individual formats will shift to serving as a flag for whether or not the theme handles format-specific data for display. If a format is not explicitly supported by the theme, there should be simple, semantic fallback output that can be hooked on to the_content. This allows the admin UI to stay consistent and able to offer data fields that don’t just disappear into nothingness, and themes can actually support post formats with just styling rather than handling format-specific data themselves.

@beaulebens is working on the fallback output on #23347, and @sabreuse is joining @rachelbaker to work on the edit form and switcher on #19570. @melchoyce has updated wireframes she’ll be linking to in a comment here. TBD: no-JS treatment, will also need to be keeping an eye on accessibility.

Also, user testing was done with core as-is, and I’ve got videos to watch and analyze for post formats using Crowd Favorite’s UI and fallback code. These are extremely revealing and are really helping us identify pain points as well as setting a baseline for measuring any improvements we hope to make.

#3-6, #post-formats

Post Formats UI Update, 1/28

@lessbloat has started new rounds of user testing – one with core as-is, and one using the Crowd Favorite code and San Kloud for the theme so that all formats are enabled. Will be reviewing and posting those when they come in.

We’re going to go ahead and get patching so we can test and iterate, and still looking for somebody interested in creating icons. @rachelbaker will be working on the form itself and switching. Would love a volunteer or two on data retrieval and theme usage functions, but will likely be looking at that myself in the next couple of days.

Office hour log

#3-6, #post-formats

Post Formats UI Update, 1/24

Apologies for the late update – helps if you actually publish the post! 🙂 We were in #wordpress-ui due to a little scheduling conflict – we’ll hopefully be back in #wordpress-dev next week, and are sorry for any confusion.

Discussion point 1: format switcher, with an example from @lessbloat: http://f.cl.ly/items/1t09071U2v1E2x2s3X2i/post-formats.png (we would use “Standard” as opposed to “Text” for that label). We will not be blocking the current behavior/ability to just start writing a post when you enter the add new screen, i.e. no forced selection of a post format before you can start writing. There is some concern about user confusion in that one might think that you can/should specify data for multiple formats in a single post if the switcher is always viewable. A viable idea seems to be that all formats are shown when you load the add new post page, which then collapse into one item (the selected one) upon a specific selection. Editing an existing post, including drafts, will only show the one format selected. Clicking on the selected format would allow for switching of the format via a to-be-determined interaction. There’s also the idea that a screen option would be provided to turn off the switcher entirely should the user so desire, just as the “Format” metabox can be turned off now.

Action: Call for icons, one per post format. No dimensions determined yet; keep in mind that 2x versions will be needed.

Discussion point 2: data structure needs, based on this really great start on wireframes by @melchoyce: https://make.wordpress.org/core/2013/01/22/post-formats-ui-update-121/#comment-7674. Proposed:

  • Media URL/embed code/shortcode (basically, text), to be shared between audio and video
  • Link URL, to be shared between link and image
  • Quote content
  • Quote source
  • Gallery, which would be a shortcode that likely needs to default to [gallery] for backcompat; would use shortcode with list of IDs for new style
  • Image, which could be an attachment ID or URL (this is separate from featured image)

Sharing data fields (storage TBD, likely post meta) would mean that switching between formats that are similar in data needs would retain that field. Action: discuss proposed data structure in comments; pros/cons of sharing fields between formats, anything missed, etc. Also discuss anything related to the wireframes; @melchoyce was working on some discussed tweaks (although she may be understandably behind due to computer theft 🙁 ), and there will likely be more that we’ll want to do.

Finally, as a tightly-related item, @wonderboymusic started work on the possibility of an HTML5 audio/video player in core: #23282. Testing and feedback more than welcome.

Full chat log

#3-6, #post-formats

Post Formats UI Update, 1/21

Still looking for wireframes/storyboards for possible UI ideas for each post format. To put it succinctly, we are looking to address discoverability, value, and usability, approximately in that order. We will at some point need to talk about how users get there in the first place, but again, for now let’s focus on the add/edit post screen. We briefly touched on what would happen to the UI if the theme doesn’t support the post format – there seems to be early consensus that the UI itself stay altered (e.g. if there were a separate input for a link URL) but there would be a warning of some sort about the lack of theme support.

A large portion of our office hour today was spent discussing a script for user testing that we can use throughout this process to help us evaluate the changes proposed. We need to use a theme that includes all ten formats, doesn’t do anything to the admin UI, and preferably does something to the front end display for each. One that I know of is San Kloud, although it has some deprecated function usage (hey @kovshenin – want to fix that? 🙂 ); other suggestions welcome.

Since the first part of the issue here is discoverability, the script needs to not hand-hold the format selection or entry process. Here’s a rough outline of what it might look like – discuss any changes in the comments, and @lessbloat will help us get testing. Some of these are particularly difficult to describe in a non-leading manner. Through each of these, it would be good to have the user narrate what they want or expect will happen for each of these formats. A little fuzzy since we’re not going to point them at the format picker and a given tester may not be used to blogging, but the more narration of thinking/expectations, the better.

  • Standard/text: Add a blog post entitled X, with the following text. Publish and view the post.
  • Aside: Add a brief blog post without a title, or what might be referred to as an “aside”. Publish and view.
  • Status: Add a blog post that represents your current status as (provide text), the way you might on a site like Facebook or Twitter. Publish and view.
  • Chat: Add a chat blog post where the content is a chat transcript (provide something to copy-paste). Publish and view.
  • Quote: Add a quote blog post with X quote attributed to Y. Publish and view.
  • Link: Add a link blog post with X text linking to [URL]. Publish and view.
  • Image: Add an image blog post and insert X image into the content editor. Publish and view.
  • Gallery: Add an image gallery blog post with X images (this probably needs step-by-step instructions for creating a gallery). Publish and view.
  • Video: Add a video blog post with [something from YouTube]. Paste the URL on its own line without linking it. Publish and view.
  • Audio: Add an audio blog post with [something from SoundCloud]. Paste the URL on its own line without linking it. Publish and view.
  • View your site. (If they chose formats, it should show the differences pretty clearly in a post archive view, hence the importance of choosing an appropriate theme.)

Full IRC chat log

#3-6, #post-formats

Post Formats UI Update, 1/17

The post formats UI feature encompasses a couple things: the add/edit UI itself, and also what this means in terms of structured data and theme usage and expectations of data. Our goal is to make post formats meaningful and usable, because let’s face it: they’re not. What can we do in the admin to help a user understand what a given post format means and how it behaves in use, and what can we do to give themes something standardized and portable when it comes to data available for display?

Had our first office hours chat today to get started – great discussion happening on a lot of fronts, especially theme compat. We identified two distinct tasks to get us through the weekend.

The first, and most crucial piece, is thinking about/identifying what users are doing when creating a post using a post format and turning that into (likely annotated) wireframes/storyboards. That is to say – what sort of UI might we want want to see on the add/edit post screen? We’ll want to think about things like what makes it easy for a user to select a format, what a user might expect to see on the edit screen for a given format (no title, or the title is de-emphasized, or what images show in that gallery, etc.), and what happens when switching between formats. Some existing examples are CF Post Formats and WordPress.com. We also shouldn’t ignore what Tumblr does for its various formats. @alexkingorg gave us a demo link for CF and @beaulebens was kind enough to take some screenshots:

Please post in a comment here with anything you come up with. Don’t worry about anything beyond the add/edit post screen for the moment. Also don’t worry about what data exactly is or is not represented – base your ideas on user workflows. The workflows and UI will heavily influence any structured data. There are also some other things to keep in the back of your mind as possible influences for anything core UI-related: how might a layout you’re imagining adapt to various screen sizes and devices, and are there pieces that can be used as the basis for more generalized core UI components.

The second piece for now is functions (probably using regex, and I know) for pulling out relevant data for post formats as appropriate, e.g. the first URL in the content for the link format, oEmbed URL or appropriate HTML element for audio/video formats, etc. What might or might not be done with the data hasn’t been determined yet, but we can at least start here. Some existing examples are the twentyeleven_url_grabber() function and what they use on WordPress.com. Nobody has claimed this task. For now, go ahead and upload any patches to the main ticket, #19570.

There are also some related existing tickets that could use testing/+1’s or patches:

  • #16047 – separate column for post format in the list table
  • #23198 – pass the post format as a class to TinyMCE
  • #15323 – list table filter by post format
  • #15514 – add to the cat-tag convertor plugin

Read the full IRC log here.

#3-6, #post-formats

Office Hours: Post Formats UI

We’ll be having our first feature team meeting in IRC tomorrow (Thursday, January 17) at 11AM Eastern/16:00 UTC. Please join us if you’re interested in participating in development of this feature, which does involve more than “just” UI, as we’re going to hit the ground running. We’ll keep that time every week, along with Mondays at 2PM Eastern/19:00 UTC. Updates will be posted here on Make/Core after each meeting. Pete Mall will be my backup lead.

#post-formats