Audio / Video support in Core

Post Formats are a big feature in WordPress 3.6. What you may not know is: there is now native support for Audio and Video in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.! There has been great support for embeds by way of WP_Embed and oEmbed providers for a while, but, if you wanted to play an MP3 from your Media Library, you had to install a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party. Supporting audio and video in core gives bands, podcasters, vloggers, et al the ability to easily and beautifully expresses themselves through sounds and moving pictures without using an external service.

How does this work?

At the core of the experience is the fantastic library, MediaElement.js. MediaElement is the facade layer that gives us maximum file support and cross-browser compatibility. While some libraries require a Flash-only solution to make your media work cross-environment, MediaElement lets you use HTML5 audio / video tags in every browser, and, only when necessary, will use a Flash or Silverlight plugin in the background to make incompatible media work. Translationtranslation The process (or result) of changing text, words, and display formatting to support another language. Also see localization, internationalization., things like this: <audio> tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) works in old IE, Windows Media files work in Chrome.

MediaElement uses the same HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. markup, regardless of playback implementation, and you can use CSSCSS Cascading Style Sheets. to skin the players.

Shortcodes

MediaElement’s great, but we don’t want to be locked in to one external library forever. Instead of using MediaElement-specific markup everywhere, we expose audio and video markup through shortcodes: [audio] and [video].

For the following scenarios:

  • I have an old post that has a video in the Media Library attached to it, and I want to use the new shortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site.: [video]
  • I have the URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org for a video, from the Media Library or external, that I want to play:
    [video src="video-source.mp4"]
  • I have a source URL and fallbacks for other HTML5-supported filetypes:
    [video width="600" height="480" mp4="source.mp4" ogv="source.ogv" webm="source.webm"]

Same goes for audio:

  • I have an old post that has an audio file in the Media Library attached to it, and I want to use the new shortcode: [audio]
  • I have the URL for an MP3, from the Media Library or external, that I want to play: [audio src="audio-source.mp3"]
  • I have a source URL and fallbacks for other HTML5-supported filetypes:
    [audio mp3="source.mp3" ogg="source.ogg" wav="source.wav"]

Shortcodes focus on the “what” and abstract the “how.” If you want to use a library that is not MediaElement, you can! Just look at what to filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.: here

Embeds

There are also new embed handlers for audio and video. Using them is easy as dropping a media link on a line by itself in the editor:

http://my.mp3s.com/cool/songs/coolest.mp3

I like this song because it is really cool!

Works for both audio and video with URLs matching the allowed (and filterable) list of extensions (see: here and here)

Adminadmin (and super admin)

Using the new post formats UIUI User interface, it is even easier to get directly at the audio and video in your Media Library. When selecting, the media modal opens to your library, filtered by media type.

Metadata

In previous versions of WP, you could upload audio and video, but we were not generating metadata like we do for images. In 3.6, using the getID3 library, we are able to extract data from audio and video like cover art, song length, artist, album, song title, genre, codec, etc. It’s pretty great. We will soon be exposing more of this data in the admin as well, along with inline previews on the Edit Media page:

Themers

Themers can get in on the action, too, using structured-post-formats in their theme (Twenty Thirteen is a great place to look). The admin gives users flexibility when associating media with a post. the_post_format_audio() and the_post_format_video() will automagically retrieve and output your media in the front end.

#audio, #media, #mediaelement, #post-formats, #shortcodes, #video

Post Formats UI Update, 3/14

As noted in The Road to 3.6 Beta 1, we’ve got quite a bit going on for post formats. Many of the tickets are in need of testing (including unit tests) and then a commit. As always, there are a few different fronts: UIUI User interface/administration, data, and parsing. Here’s where we are with each, and what needs to get done. There’s a large variety of tasks here, and we are seeking contributors to help 🙂

Continue reading

#3-6, #post-formats

Post Formats UI Update, 2/21

First run commit, mostly for soaking of functionality, went into trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. on Monday. This includes fallback output for theme compat and fields in the adminadmin (and super admin). The admin UIUI User interface does not represent anything final – especially of note is the tabs, which are essentially the way they are in order to reuse CSSCSS Cascading Style Sheets. and avoid adding temporary code to coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress..

Continue reading

#3-6, #post-formats

Post Formats UI Update, 2/4

Not a whole lot to discuss today – working away at patches on #19570 (edit form UIUI User interface) and #23347 (theme compat). Could use a patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing. on #15490 (retrieval of oEmbed data and previewing, which I suspect will become relevant to the edit form UI) and testing of #23282 (audio/video shortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. and player) and #16047 (list table column).

#3-6, #post-formats

Post Formats UI Update, 1/31

Had really good conversations in Wednesday’s dev chat and continued in today’s office hours. Seems like we’ve got a really good direction. All the UIUI User interface will be available all the time and theme support for individual formats will shift to serving as a flag for whether or not the theme handles format-specific data for display. If a format is not explicitly supported by the theme, there should be simple, semantic fallback output that can be hooked on to the_content. This allows the adminadmin (and super admin) UI to stay consistent and able to offer data fields that don’t just disappear into nothingness, and themes can actually support post formats with just styling rather than handling format-specific data themselves.

@beaulebens is working on the fallback output on #23347, and @sabreuse is joining @rachelbaker to work on the edit form and switcher on #19570. @melchoyce has updated wireframes she’ll be linking to in a comment here. TBD: no-JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors. treatment, will also need to be keeping an eye on accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility).

Also, user testing was done with core as-is, and I’ve got videos to watch and analyze for post formats using Crowd Favorite’s UI and fallback code. These are extremely revealing and are really helping us identify pain points as well as setting a baseline for measuring any improvements we hope to make.

#3-6, #post-formats

Post Formats UI Update, 1/28

@lessbloat has started new rounds of user testing – one with coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. as-is, and one using the Crowd Favorite code and San Kloud for the theme so that all formats are enabled. Will be reviewing and posting those when they come in.

We’re going to go ahead and get patching so we can test and iterate, and still looking for somebody interested in creating icons. @rachelbaker will be working on the form itself and switching. Would love a volunteer or two on data retrieval and theme usage functions, but will likely be looking at that myself in the next couple of days.

Office hour log

#3-6, #post-formats

Post Formats UI Update, 1/24

Apologies for the late update – helps if you actually publish the post! 🙂 We were in #wordpress-ui due to a little scheduling conflictconflict A conflict occurs when a patch changes code that was modified after the patch was created. These patches are considered stale, and will require a refresh of the changes before it can be applied, or the conflicts will need to be resolved. – we’ll hopefully be back in #wordpress-dev next week, and are sorry for any confusion.

Discussion point 1: format switcher, with an example from @lessbloat: http://f.cl.ly/items/1t09071U2v1E2x2s3X2i/post-formats.png (we would use “Standard” as opposed to “Text” for that label). We will not be blocking the current behavior/ability to just start writing a post when you enter the add new screen, i.e. no forced selection of a post format before you can start writing. There is some concern about user confusion in that one might think that you can/should specify data for multiple formats in a single post if the switcher is always viewable. A viable idea seems to be that all formats are shown when you load the add new post page, which then collapse into one item (the selected one) upon a specific selection. Editing an existing post, including drafts, will only show the one format selected. Clicking on the selected format would allow for switching of the format via a to-be-determined interaction. There’s also the idea that a screen option would be provided to turn off the switcher entirely should the user so desire, just as the “Format” metaboxMetabox A post metabox is a draggable box shown on the post editing screen. Its purpose is to allow the user to select or enter information in addition to the main post content. This information should be related to the post in some way. can be turned off now.

Action: Call for icons, one per post format. No dimensions determined yet; keep in mind that 2x versions will be needed.

Discussion point 2: data structure needs, based on this really great start on wireframes by @melchoyce: https://make.wordpress.org/core/2013/01/22/post-formats-ui-update-121/#comment-7674. Proposed:

  • Media URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org/embed code/shortcodeShortcode A shortcode is a placeholder used within a WordPress post, page, or widget to insert a form or function generated by a plugin in a specific location on your site. (basically, text), to be shared between audio and video
  • Link URL, to be shared between link and image
  • Quote content
  • Quote source
  • Gallery, which would be a shortcode that likely needs to default to [gallery] for backcompat; would use shortcode with list of IDs for new style
  • Image, which could be an attachment ID or URL (this is separate from featured imageFeatured image A featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts.)

Sharing data fields (storage TBD, likely post metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress.) would mean that switching between formats that are similar in data needs would retain that field. Action: discuss proposed data structure in comments; pros/cons of sharing fields between formats, anything missed, etc. Also discuss anything related to the wireframes; @melchoyce was working on some discussed tweaks (although she may be understandably behind due to computer theft 🙁 ), and there will likely be more that we’ll want to do.

Finally, as a tightly-related item, @wonderboymusic started work on the possibility of an HTML5 audio/video player in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.: #23282. Testing and feedback more than welcome.

Full chat log

#3-6, #post-formats

Post Formats UI Update, 1/21

Still looking for wireframes/storyboards for possible UIUI User interface ideas for each post format. To put it succinctly, we are looking to address discoverability, value, and usability, approximately in that order. We will at some point need to talk about how users get there in the first place, but again, for now let’s focus on the add/edit post screen. We briefly touched on what would happen to the UI if the theme doesn’t support the post format – there seems to be early consensus that the UI itself stay altered (e.g. if there were a separate input for a link URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org) but there would be a warning of some sort about the lack of theme support.

A large portion of our office hour today was spent discussing a script for user testing that we can use throughout this process to help us evaluate the changes proposed. We need to use a theme that includes all ten formats, doesn’t do anything to the adminadmin (and super admin) UI, and preferably does something to the front end display for each. One that I know of is San Kloud, although it has some deprecated function usage (hey @kovshenin – want to fix that? 🙂 ); other suggestions welcome.

Since the first part of the issue here is discoverability, the script needs to not hand-hold the format selection or entry process. Here’s a rough outline of what it might look like – discuss any changes in the comments, and @lessbloat will help us get testing. Some of these are particularly difficult to describe in a non-leading manner. Through each of these, it would be good to have the user narrate what they want or expect will happen for each of these formats. A little fuzzy since we’re not going to point them at the format picker and a given tester may not be used to blogging, but the more narration of thinking/expectations, the better.

  • Standard/text: Add a blogblog (versus network, site) post entitled X, with the following text. Publish and view the post.
  • Aside: Add a brief blog post without a title, or what might be referred to as an “aside”. Publish and view.
  • Status: Add a blog post that represents your current status as (provide text), the way you might on a site like Facebook or Twitter. Publish and view.
  • Chat: Add a chat blog post where the content is a chat transcript (provide something to copy-paste). Publish and view.
  • Quote: Add a quote blog post with X quote attributed to Y. Publish and view.
  • Link: Add a link blog post with X text linking to [URL]. Publish and view.
  • Image: Add an image blog post and insert X image into the content editor. Publish and view.
  • Gallery: Add an image gallery blog post with X images (this probably needs step-by-step instructions for creating a gallery). Publish and view.
  • Video: Add a video blog post with [something from YouTube]. Paste the URL on its own line without linking it. Publish and view.
  • Audio: Add an audio blog post with [something from SoundCloud]. Paste the URL on its own line without linking it. Publish and view.
  • View your site. (If they chose formats, it should show the differences pretty clearly in a post archive view, hence the importance of choosing an appropriate theme.)

Full IRC chat log

#3-6, #post-formats

Post Formats UI Update, 1/17

The post formats UIUI User interface feature encompasses a couple things: the add/edit UI itself, and also what this means in terms of structured data and theme usage and expectations of data. Our goal is to make post formats meaningful and usable, because let’s face it: they’re not. What can we do in the adminadmin (and super admin) to help a user understand what a given post format means and how it behaves in use, and what can we do to give themes something standardized and portable when it comes to data available for display?

Had our first office hours chat today to get started – great discussion happening on a lot of fronts, especially theme compat. We identified two distinct tasks to get us through the weekend.

The first, and most crucial piece, is thinking about/identifying what users are doing when creating a post using a post format and turning that into (likely annotated) wireframes/storyboards. That is to say – what sort of UI might we want want to see on the add/edit post screen? We’ll want to think about things like what makes it easy for a user to select a format, what a user might expect to see on the edit screen for a given format (no title, or the title is de-emphasized, or what images show in that gallery, etc.), and what happens when switching between formats. Some existing examples are CF Post Formats and WordPress.comWordPress.com An online implementation of WordPress code that lets you immediately access a new WordPress environment to publish your content. WordPress.com is a private company owned by Automattic that hosts the largest multisite in the world. This is arguably the best place to start blogging if you have never touched WordPress before. https://wordpress.com/. We also shouldn’t ignore what Tumblr does for its various formats. @alexkingorg gave us a demo link for CF and @beaulebens was kind enough to take some screenshots:

Please post in a comment here with anything you come up with. Don’t worry about anything beyond the add/edit post screen for the moment. Also don’t worry about what data exactly is or is not represented – base your ideas on user workflows. The workflows and UI will heavily influence any structured data. There are also some other things to keep in the back of your mind as possible influences for anything coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. UI-related: how might a layout you’re imagining adapt to various screen sizes and devices, and are there pieces that can be used as the basis for more generalized core UI components.

The second piece for now is functions (probably using regex, and I know) for pulling out relevant data for post formats as appropriate, e.g. the first URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org in the content for the link format, oEmbed URL or appropriate HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. element for audio/video formats, etc. What might or might not be done with the data hasn’t been determined yet, but we can at least start here. Some existing examples are the twentyeleven_url_grabber() function and what they use on WordPress.com. Nobody has claimed this task. For now, go ahead and upload any patches to the main ticketticket Created for both bug reports and feature development on the bug tracker., #19570.

There are also some related existing tickets that could use testing/+1’s or patches:

  • #16047 – separate column for post format in the list table
  • #23198 – pass the post format as a class to TinyMCE
  • #15323 – list table filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. by post format
  • #15514 – add to the cat-tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) convertor pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party

Read the full IRC log here.

#3-6, #post-formats

Office Hours: Post Formats UI

We’ll be having our first feature team meeting in IRCIRC Internet Relay Chat, a network where users can have conversations online. IRC channels are used widely by open source projects, and by WordPress. The primary WordPress channels are #wordpress and #wordpress-dev, on irc.freenode.net. tomorrow (Thursday, January 17) at 11AM Eastern/16:00 UTC. Please join us if you’re interested in participating in development of this feature, which does involve more than “just” UIUI User interface, as we’re going to hit the ground running. We’ll keep that time every week, along with Mondays at 2PM Eastern/19:00 UTC. Updates will be posted here on Make/CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. after each meeting. Pete Mall will be my backup lead.

#post-formats