WordPress.org

Make WordPress Core

Tagged: media Toggle Comment Threads | Keyboard Shortcuts

  • Joe McGill 11:35 pm on August 25, 2015 Permalink |
    Tags: media, respimg   

    Update: Responsive Image Support for Core 

    The responsive image team has been meeting regularly for a while. After our meeting earlier this week, we realized that make/core needs an update on what’s been going on with the RICG (Responsive Images Community Group) feature plugin, as well as to request feedback on a few questions.

    Our meetings are in #feature-respimg each Friday at 1900 UTC, so please come and chat to give feedback or if you’re interested in helping out!

    Background

    Several years ago, a ragtag group of web professionals identified a need for new HTML markup which would allow developers to declare multiple sources for an image—allowing devices to select the image source that was most appropriate for its own capabilities. Fast forward to today and all major browsers have either implemented these new tools or currently have them under consideration for development.

    With that as background, the RICG has been working on an Official WordPress Feature Plugin™ to test the viability of adding responsive image support natively into WordPress. Specifically, our aim is to automatically add srcset (using w descriptors) and sizes attributes to the image markup generated by WordPress. According to the WordPress.org plugin directory, there are over 10k active installs, so we’ve definitely seen an interest in this functionality.

    There are two main pieces of functionality included in the plugin, which can be considered separately for inclusion in core:

    1. Logic for producing responsive image markup
    2. Advanced image compression (via ImageMagick)

    Responsive Image Markup

    There is a lot to consider when proposing a change to the way WordPress outputs image markup, so I want to be clear about some of our assumptions going in:

    • Responsive image support should be added ‘invisibly’ without introducing new settings for users to worry about.
    • WordPress, out of the box, should only deal with resolution switching, and not the art direction use case for now. In other words, we would not be adding any API or admin UI for outputting different cropped images at specific breakpoints. (For more information about use cases and all things related to responsive images, I’d recommend reading the terrific Responsive Image 101 series by Jason Grigsby).
    • Provide this functionality using default and available user-defined sizes (via add_image_size()) for source sets rather than creating an additional set of crops. This choice is based on early feedback from Nacin regarding file-count concerns on some shared hosts.
    • Provide filter hooks so theme/plugin authors can extend/override defaults.
    • All sizes of an image (i.e., _wp_attachment_metadata['sizes']) with the same aspect ratio are resized versions of the same image, not custom art directed crops. This assumption has been okay so far, but there may be be plugins that replace the default image sizes with art directed crops that maintain the same aspect ratio. We’ll need to determine how to handle these cases.

    Questions to Consider

    1. How should we handle markup embedded in post content?
      • Currently, we embed the srcset attribute directly into posts with sizes added as a data attribute to make it easier for theme authors to filter the sizes attribute later. It’s tricky to decide when it’s appropriate to add layout relative markup to the database, but WordPress is currently doing this to a certain extent by adding width/height attributes to images, so this may be the best solution for now.
      • Instead, a more elegant solution would be to filter the content on output. This would avoid saving layout markup in the database, and extend support to posts with images that were published before the feature became available. We have a proof of concept but are unsure if adding another filter to the_content is appropriate. Confirmation either way on this question would help us move forward.
    2. Should we support srcset and sizes in older browsers?
      • The plugin includes the picturefill.js polyfill, which provides support for older browsers, but would be a new dependency for core.
      • We could view srcset and sizes as a progressive enhancement and only provide support in WordPress for newer browsers. In that case, we could drop the polyfill and save WordPress an extra JS dependency. Note that this polyfill is written by the same people writing and implementing the spec. We consider it to be very reliable.
    3. Should we turn responsive image support on by default?
      • “Decisions not options.” We propose that responsive images are enabled by default in core, with filters provided for disabling or modifying the feature.
      • If core does not want responsive images enabled by default, they could be enabled through a current_theme_supports() flag. Themes would have to “opt-in” to the feature.

    Advanced Image Compression

    The second potential enhancement introduced through our plugin is an improvement to the default ImageMagick compression settings currently being used in core. RICG contributor Dave Newton has done great research on the best compression settings for ImageMagick, and included them as an opt-in option within the plugin.

    The updated settings are absolutely killer when there are sufficient CPU and memory resources on the host server. In our trials, file sizes have decreased by >50% compared to the default core settings.

    However, on limited servers, these new settings could cause problems. We are iterating on them to find the right balance between the file-size savings and the CPU resources required to process large files.

    Final Notes

    We need your help!

    New features need lots of feedback and testing. Help us test these enhancements by installing the latest version of the plugin from WordPress.org.

    Be sure to enable advanced image compression and tell us how it does with your setup so we can gather more feedback.

    If you know of plugins that heavily modify or interact with custom image sizes or art-directed crops, please leave a comment below so we can test it with this feature.

    Have thoughts on the questions above? Let us know in the comments!

    Want to get involved? We meet each week in #feature-respimg on Friday at 1900 UTC.

     
    • Ahmad Awais 1:08 am on August 26, 2015 Permalink | Log in to Reply

      Will join the next meeting. Let’s do it.

    • bravokeyl 1:58 am on August 26, 2015 Permalink | Log in to Reply

      It’s a good one , but is “ element getting traction from specifications ?

      • wilto 2:49 pm on August 26, 2015 Permalink | Log in to Reply

        Yes indeed! All of the markup patterns used by the plugin are part of the HTML Living Standard:
        https://html.spec.whatwg.org/multipage/embedded-content.html#embedded-content

        While the plugin is using Picturefill for the sake of older browsers, all modern browser support this markup natively to some extent—current versions of Firefox and Chrome support it fully, while Safari and Microsoft Edge have partial support. Picturefill polyfills these features in a granular way, so anything that can work natively will, while anything without native support will be shimmed.

    • Robert Chapin 2:10 am on August 26, 2015 Permalink | Log in to Reply

      For the HiDPI Gravatars plugin, I’ve been referencing caniuse.com with the “Usage Relative” option selected. We really are at the point now where Internet Explorer 11 is the only significant browser without support for srcset. Decide accordingly. Is it really worth adding core bloat to support IE 11? Would it benefit anyone other than Surface users? Are the accessibility benefits for desktop just as easily acheived in Chrome?

      • Joe McGill 2:00 pm on August 26, 2015 Permalink | Log in to Reply

        This is almost true. As of today, Safari only supports `srcset` with x descriptors so the polyfill is still needed to support w descriptors and `sizes`. There are signs that the next version will include full support, but that is currently not the case.

    • Michael Arestad 2:21 am on August 26, 2015 Permalink | Log in to Reply

      > 2. Should we support srcset and sizes in older browsers?

      No. No need. this can gracefully degrade.

      > 3. Should we turn responsive image support on by default?

      This is definitely the hardest question and might answer itself during larger-scale testing. Going forward, it would make sense to unless it breaks a bunch of themes that do crop the attachment images (or do something else funky).

      • Joe McGill 2:07 pm on August 26, 2015 Permalink | Log in to Reply

        Thanks Michael,

        As a counter argument for supporting srcset and sizes in older browsers: with many older devices being in use around the world, I wonder if the addition of the polyfill—at just over 11kb minified—is worth the potential bandwidth savings that would be gained by potentially loading smaller image sources.

    • padams 6:25 am on August 26, 2015 Permalink | Log in to Reply

      Responsive images are probably not going to be very useful unless the theme itself has a responsive design. Did you guys think about making themes declare support for responsive images instead of making it a global on/off setting?

      • tevko 1:13 am on August 27, 2015 Permalink | Log in to Reply

        Hey padams, since we’re using the srcset attribute with sizes, responsive images will help regardless of fixed width or fluid designs. This is because the srcset / sizes combination works to deliver the best image size based on the users screen resolution and viewport width. This means that two different devices at the same width could get a different image, simply because their screen resolutions are different. This is great for the user because it means that their device will get the best looking image regardless of viewport size.

    • Monika 6:42 am on August 26, 2015 Permalink | Log in to Reply

      >> 3. Should we turn responsive image support on by default?

      >>>If core does not want responsive images enabled by default, they could be enabled through a current_theme_supports() flag. Themes would have to “opt-in” to the feature.

      yes please!

      Why:
      1. RICG This plugin works by including all available image sizes for each image upload.

      Most of the time we don’t need “all” image sizes , thats why I preferr
      https://wordpress.org/plugins/responsify-wp/

      it is easier to use as RICG if I don’t want all image sizes

      Feedback: If I’m using: ‘advanced-image-compression

      my upload failes
      256php memory

    • kuzvac 7:11 am on August 26, 2015 Permalink | Log in to Reply

      Please use “picture” element instead of srcset! Picture element already have backward compatibility to older browsers. https://dev.opera.com/articles/native-responsive-images/
      And use cases https://dev.opera.com/articles/responsive-images/
      And progressive jpg, god, yes.

      • wilto 2:54 pm on August 26, 2015 Permalink | Log in to Reply

        For a “fully automated” situation like this one, we’re apt to get a much better result using `srcset`/`sizes`. `picture` is intended for “art direction” use cases, where there’s a need to set explicit and very specific breakpoints based on the viewport, with the image sources themselves varying slightly. All the `srcset` syntaxes, however, allow the browser to choose the best fit from a list of sources that are identical (apart from their dimensions). `srcset` factor’s in the user’s display density, the size of the image _in the layout_ (rather than based on the viewport), and soon additional factors like a user preference or a bandwidth cap. Since WP will be generating all the different cuts of an image, we end up with something completely seamless: smaller images for the user and no wall of breakpoint decisions for the image uploader.

        And no worries about backwards compatibility—that’s covered with `srcset` as well, either via Picturefill or via the old-fashioned `src` attribute on `img`.

    • Brad Touesnard 11:48 am on August 26, 2015 Permalink | Log in to Reply

      > 2. Should we support srcset and sizes in older browsers?

      I think by default they should not be supported in older browsers, but it should be easy to just install & activate an official plugin to enable them for older browsers.

      > 3. Should we turn responsive image support on by default?

      I suggested the `current_theme_supports()` flag on Slack. I think it should be up to the theme to turn on responsive image support. If the theme is designed for WordPress’ current image resizing it’s not very likely to work well with responsive images.

      I’d like to see the image compression stuff spun off into it’s own independent plugin. Is there a reason it is lumped in with the responsive image stuff?

      • tevko 1:31 am on August 27, 2015 Permalink | Log in to Reply

        Hey Brad, thanks for the feedback.

        Regarding your first point, seeing that RICG Responsive Images is the official plugin, and that plugin provides fallback support, it would seem natural to assume that if integrated into WP Core, fallback support would also be provided.

        Additionally, being an interfaceless plugin that runs behind the scenes, it wouldn’t be readily apparent to users that something else would need to be installed in order to provide fallback support. I think bundling a small polyfill in with the feature would make it easiest on all users. The ability to dequeue the polyfill also exists and has been documented here – https://github.com/ResponsiveImagesCG/wp-tevko-responsive-images#tevkori_get_srcset_string-id-size-

    • John Huebner 12:36 pm on August 26, 2015 Permalink | Log in to Reply

      > In other words, we would not be adding any API or admin UI for outputting different cropped images at specific breakpoints.

      Just my thoughts on this one aspect. While it would be great to have something built into WP automatically handle responsive images, without the ability to specify different images this will not be of much value in the work that I’m required to do. Using the automatically generated images will only be of use to the most generic users. I don’t think I have a single client that would be happy with this model, which is why I don’t use the plugin mentioned.

      • wilto 3:01 pm on August 26, 2015 Permalink | Log in to Reply

        We’re totally agreed that the “art direction” use case covered by the `picture` element—and the accompanying UI changes that would have to go into uploading images—is a valuable one. It’s in the planning stages now.

        However, just like you said, this _is_ a feature of use to the most generic users: anyone putting together a responsive theme that contains large images. If you’ve got visitors to your site using small viewports or low-density displays and you’ve got a theme that contains large or high-resolution images, adding this feature to core ensures that you’re not serving massive images to small viewports or high-density images to low-density displays—situations where the user will see no benefit, but be forced to incur an additional cost.

        It’s better to think of this as a behind-the-scenes enhancement to images, rather than a situation where your client will need to pick and choose breakpoints. Rolling out these features on FT.com resulted in a 66% reduction in image weight*. On my current project we’re looking at an ~80% reduction on 320px, low-density displays, without any changes to the uploader workflow.

    • Nicolas Juen 3:32 pm on August 26, 2015 Permalink | Log in to Reply

      Hi,
      I’m glad this feature is coming to the core, in my agency we are already using a custom solution for this consisting of this plugin (https://github.com/asadowski10/advanced-responsive-images) + WP_Thumb.
      The difference here is that the frontend developper is defining locations on json file, and associate image sizes + breakpoints wich are defined in one other file.
      Then on post_thumbnail function we call with one arg (data-location) with the location defined on json file. This allow us to handle multiple display cases, like in the slider, content, widget area etc.

      Is this conception have been tough ? Is this core functionnality will be hackable so we can do like this ?
      We are using picture fill too but it’s not compatible with lazyload, some libraries do and use the picture element, wich is pretty different from srcset.

    • Ricard Torres 8:18 pm on August 26, 2015 Permalink | Log in to Reply

      One of the top plugins out there that deals with images: https://wordpress.org/plugins/regenerate-thumbnails/

    • Phil Smart 11:57 pm on August 26, 2015 Permalink | Log in to Reply

      I’m just going to weight in quickly and say that I actually like the idea of what the Fusion guys are exploring with their plugin, Image Shortcake (http://fusion.net/story/144755/introducing-image-shortcake-v0-1-0/).

      Their basic approach is to add images to TinyMCE as shortcodes – instead of direct markup – giving them the flexibility to dynamically implement whatever specific markup they need.

      I know it’s a stepping a little deeper than straight up responsive image support, but storing images as shortcodes does seem to be a great starting point for then folding in (and easily updating) necessary markup.

    • Morten Rand-Hendriksen 5:14 pm on August 27, 2015 Permalink | Log in to Reply

      Quick answers:

      1. As you probably know from my blog posts, I am a proponent of the filter-on-output solution. For responsive images to work as effectively as the spec allows, theme and plugin devs have to be able to define the actual display sizes of images through variables. For that to work, the HTML output must be filtered based on current theme and plugins. This will cause some issues with caching services and CDNs, but I think that’s a minor concern. The one thing I hope can be avoided is the blanket approach of sizes=”100w, [original_image_size]”

      2. Backwards compatibility within reason is impoetatn because those that will benefit the most from this development (people on slow / low bandwidth / expensive connections and older devices) often use older browsers.

      3. Yes, turn it on by default. This should be invisible and transparent. We have a major opportunity here to use WordPress to push web standards forward, and that cannonly happen if we make some serious commitments like this.

      Great work to everyone who has been working on this. I am very excited about this plugin and its potential.

  • Nick Halsey 3:30 pm on July 16, 2015 Permalink |
    Tags: , , , , media,   

    New Customizer Media Controls in 4.3 and 4.2 

    Over the past several releases, the Customizer API has drastically improved its built-in media controls, empowering developers to leverage the power of the WordPress media management experience in themes and plugins with ease. WordPress 4.1 refactored the image and upload controls to leverage the media modal for the first time (see #21483). 4.2 abstracted this functionality to a new base media control class. And now in WordPress 4.3, we’ve added a control for cropped images. In this post, I’ll outline the recent changes in Customizer media controls and explain the differences between the available controls.

    WP_Customize_Media_Control

    Before WordPress 4.2, all Customizer media controls saved the file url to their corresponding settings. While this facilitates quick access when using the value of the setting in themes and plugins, it makes it more difficult to access other information about that attachment, such as its title/caption, mime type, or in the case of images, accessing specific image sizes.

    WP_Customize_Media_Control was introduced to allow this paradigm to change while maintaining backwards compatibility for the existing WP_Customize_Upload_Control and WP_Customize_Image_Control, which now extend the media control class (see #29215). The media control will save the attachment id for the selected media file to the Customizer setting, rather than the file URL. However, note that the default value of the setting must still be a URL, since a default attachment id doesn’t really make sense.

    The media control can be used for any type of media, be it an image, audio, video, PDF document, or any other file format that your server supports. To restrict a media control instance to a particular type of content, use the mime_type parameter when adding the control:

    $wp_customize->add_control( new WP_Customize_Media_Control( $wp_customize, 'audio_control', array(
    	'label' => __( 'Media Control (audio)' ),
    	'section' => 'media',
    	'mime_type' => 'audio',
    ) ) );

    When working with a setting corresponding with a media control, the sanitize_callback should generally be absint(), since a numerical id is expected. When using get_option() and get_theme_mod(), functions such as wp_get_attachment_url(), wp_get_attachment_image(), wp_get_attachment_image_src(), and even get_post() are useful depending on your needs, with each function taking the attachment id (value of the setting) as a parameter.

    The full power of WP_Customize_Media_Control is realized when the control is extended to implement additional custom functionality in a custom child control. WP_Customize_Cropped_Image_Control is a great example of this in core. The core Customizer control classes provide several working examples of this; see wp-includes/class-wp-customize-control.php and wp-admin/js/customize-controls.js.

    WP_Customize_Cropped_Image_Control

    New in WordPress 4.3, WP_Customize_Cropped_Image_Control extracts functionality from the header image control to allow an image to be cropped to specific dimensions (see #29211). This offers a better user experience than automatic cropping in many cases when images of a certain size or aspect ratio are required in themes and plugins. In core, the new site icon feature relies heavily on the cropped-image control, implementing a child custom control to add additional site-icon-specific functionality.

    The cropped image control comes with four custom parameters in addition to those available in the media control. These are used to specify the required (or recommended) image dimensions, as well as specifying whether alternative dimensions are allowed (the flex options). Here’s a typical usage when adding a cropped-image control:

    $wp_customize->add_control( new WP_Customize_Cropped_Image_Control( $wp_customize, 'cropped_image', array(
    	'section'     => 'background_image',
    	'label'       => __( 'Croppable Image' ),
    	'flex_width'  => true, // Allow any width, making the specified value recommended. False by default.
    	'flex_height' => false, // Require the resulting image to be exactly as tall as the height attribute (default).
    	'width'       => 1920,
    	'height'      => 1080,
    ) ) );
    

    The cropped-image control creates a child attachment of the original image attachment object for the cropped image, preserving the original version. The cropped-image attachment is given a context based on the control id (with _ replaced by -). The core control doesn’t currently use this, but it could be leveraged to query for previously-cropped images for a specific control to add a library feature in the future or in child controls. Be mindful that a version of the control id is stored in the database for cropped image attachments.

    As with the media control, the cropped-image control saves the attachment id instead of the image URL. This can be useful for querying specific sizes of the image, but you’ll typically want the full size image at the cropped dimension. wp_get_attachment_image_src( absint( get_option( 'cropped_image_setting' ) ) ) should do the trick if that’s the case, when outputting the value of the setting.

     
    • Maria Antonietta Perna 4:50 pm on July 16, 2015 Permalink | Log in to Reply

      Great news! The WP Customizer keeps getting better and better.

    • Schwarttzy 5:15 pm on August 10, 2015 Permalink | Log in to Reply

      We should really create an easy to use sanitize function for “choices” so that only options in the list are able to be sanitized through. Instead of me writing a custom function for every single one that I want to do that on.

    • Peter Elmered 8:03 pm on August 12, 2015 Permalink | Log in to Reply

      Something seams to be wrong with the cropped-image control example. For me it generates the following error:

      Fatal error: Call to a member function check_capabilities() on null in /var/www/site/public/wp/wp-includes/class-wp-customize-control.php on line 283

      I use WordPress 4.3-RC2-33611

      • Peter Elmered 10:02 am on August 13, 2015 Permalink | Log in to Reply

        I managed to solve this, but how do I get the image that i cropped? I can see that the cropped image is created, but I can’t find any way to get the image URL for the cropped image from the returned ID.

        This code only gives me the standard thumbnail size, and not the cropped image:
        wp_get_attachment_image_src( absint( get_option( ‘cropped_image_setting’ ) ) )

        I can’t find anything that indicates support for this in /wp-inlucdes/media.php either.

    • smartpixels 3:55 pm on August 14, 2015 Permalink | Log in to Reply

      If you are using the Customizer JS API are we expected to use AJAX to get image URL’s if we are using the media control and storing ID’S and not URL’s

  • Ryan Boren 12:01 am on July 15, 2015 Permalink |
    Tags: , bubbles, , content-overrun, , edit-site, , , media, , , network-admin, right-now, ,   

    Today in the Nightly: Site icons in the customizer, editor patterns, more accessible comment bubbles, row toggle focus styling 

    Install the nightly, and try out this fresh batch of shiny.

    Site Icons in the Customizer

    I’ve long wanted site icons in the customizer alongside site title and tagline. The identity information that I always want to edit when first setting up a site are now all together in the customizer.

    For more visuals, see these visual records.

    See #16434.

    Editor Patterns

    Create bulleted lists, ordered lists, and blockquotes using markdown like patterns. I find this particularly handy on phones when the editor toolbar is offscreen.

    Screen Shot 2015-07-14 at 4.39.12 PM

    See #31441.

    Better focus styling for list table row toggles

    See #32395.

    Better accessibility and design for the comments bubble

    The comments columns in our list tables were among the most confusing for screen reader users. Accessibility and visuals are now improved.

    See #32152.

    Eliminate content overruns on small screens

    An audit of content overruns on small screens resulted in many fixes.

    After:

    Before:

    See #32846.

    Styling improvements on small screens for Right Now in the network admin

    See #32962.

    Improved header information in Network Admin Edit Site tabs

    • Use the site’s name rather than URL in the Edit Site header.
    • Provide “Visit” and “Dashboard” links for the site on all tabs.

    After:

    Before:

    See #32525.

    Disambiguate “Automatically add new top-level pages to this menu”

    In the customizer, a menu’s auto-add pages option is now separated from the preceding menu location checkboxes.

    See #32820.

     Passwords UI Improvements

    Passwords received a couple of improvements. The show/hide toggles look better, and passwords ui is on the install screen. Passwords on the install screen still needs a little more flow work.

    See #32589 and #32925.

    For more visuals, see these visual records.

    Reduce link noise in media library list view

    This is visually subtle but removes confusion for screen readers.

    KL_7dmW58c

    See #32254.

     

    Previously: Today in the Nightly: Customize in the Toolbar, Passwords UI, List Tables on Phones, Dashicons

     
  • Scott Taylor 9:54 pm on January 22, 2015 Permalink
    Tags: , , media   

    The Case for JS Modules 

    I originally posted some of this content here: Split javascript files in media into modules

    The patch on that ticket breaks up the Backbone classes in media-models.js, media-views.js, media-audiovideo.js, and media-grid.js into modules and injects them via Browserify on build/watch into a built file. Let’s start at the beginning.

    Brain overload

    Files that are 1000s of lines long are hard to consume. We try to alleviate this by adding copious amounts of docs. Still, it’s a lot to look at. Ideally, we would break our files into smaller modules and then somehow join them together in a build process.

    It is common practice to serve (sometimes very large) minified files for JS and CSS that have concatenated many smaller files together and uglify’d (minified/obfuscated) them. It is no longer common or best practice to develop with huge files. We can learn a lot from emerging front end development trends, especially those from the Node/NPM community. In some cases, we can even share code.

    We’ll use Media as the main culprit, but this could apply to any “manifest” – a term I use to describe files that contain the entire public API for a feature. Something like media-views.js, it might be nice to bounce from view to view in the same file, provided you know exactly what you are looking at, what depends on what, etc.

    I have found, it is completely overwhelming for almost everyone. It would be great if each discreet piece could be viewed in isolation with its dependencies clearly stated.

    There are many ways to accomplish the splitting of large files. I want to focus on 2 of the most common.

    Vocabulary

    Backbone is one of a growing number of MV* frameworks for JavaScript. A large majority of the code related to media either belongs to a handful of Models or to the increasingly large library of Views and View Templates.

    Views are the building blocks for the presentation of Media (you know, “the Media Modal” or 4.0’s “Media Grid”).

    The main canvas on which these Views are stitched together are called Frames, which are themselves Views – tilting our use of Backbone more towards MVP, P standing for Presenter.

    We have Controllers, which are called States, but they belong to a Frame (Presenter! also a View!), so anyways…. for now….

    When we create new UIs, we are more than likely adding new Views/Templates, or updating existing Views.

    If we wanted to move from one large file to many files that each contain a class, we would create Modules.

    Grunt is a task runner. We use Grunt to build our src directory into our build directory.

    Require

    Require is a great tool for converting AMD modules into built files. Require leans on Dependency Injection in its syntax:

    define([
        'models/taco',
        'models/burrito',
        'controllers/meal'
    ], function (Taco, Burrito, Meal) {
        var Dinner = Meal.extend({
            // taco-related code
        });
        return Dinner;
    });
    

    This syntax works great, unless you have way more dependencies. Refactoring code could unwind a module that has a lot of dependencies, but if you are just trying to convert legacy classes into a module, Require starts to get a little weird. Concrete example: Frames have a TON of dependencies.

    Require becomes a Grunt task to make one big file by recursing the dependency tree in an initial manifest. Require, by default, loads JS asynchronously, which can cause race conditions with plugins or themes that expect code to be registered on $(document).ready() or window.onload.

    Require works even if you don’t build via Grunt.

    Browserify

    Browserify is a tool that allows you to use Node-style modules and run them in a browser without changing from the Node syntax. Browserify requires a build for this to work.

    Using our example from above, this is the syntax for Browserify:

    var Taco = require( './models/taco.js' ),
        Burrito = require( './models/burrito.js' ),
        Meal = require( './controllers/meal.js' ),
        Dinner;
    
    Dinner = Meal.extend({
        // taco-related code
    });
    
    module.exports = Dinner;
    

    Browserify leans more towards the Service Locator pattern.

    Browserify scans the abstract syntax tree (AST) of your JS code to compile dependencies. Your modules themselves get wrapped in their own scope like so:

    (function (require, module, exports) { 
        .....YOUR_MODULE..... 
    })

    After spending a lot of time messing around with both: I think we should use Browserify.

    Converting “Legacy” Code

    The media JS code is some of the most “modern” code in WordPress, but it still clunkily lives in huge files. To convert the code into modules, we need to make a lot of individual files (one for each Backbone class).

    We also need to make sure we maintain the existing wp.media namespaces for backwards compatibility. We don’t want any existing functionality to change, we just want to build the files differently.

    Even though the code is defined differently, wrapped in a new scope, and looks different when “built”, we can still maintain our current API design: what is publicly accessible now will remain publicly accessible.

    In the patch

    Disclaimer: this patch is for experimentation only. It will go stale probably before this post is published. It works, but it is only a playground for now. If this moves forward, it will be a laborious Subversion process to create a bunch of new files.

    I have added a folder to wp-includes/js, media, that contains the modules and the built manifests. My patch adjusts script-loader.php to use these new paths.

    media contains the following files/folders:

    controllers/
    models/
    routers/
    utils/
    views/ (with another round of subfolders)
    audio-video.manifest.js
    grid.manifest.js
    models.manifest.js
    views.manifest.js
    

    The build pipeline

    If you are following along with that patch and want to see this in action, run in the project root:

    npm install
    

    Afterwards, run:

    grunt watch
    

    *.manifest.js files get built into *.js files when you change a file in media/*, provided you are running the grunt watch task. The watcher will automatically call browserify:media and uglify:media when those files change. This allows you to run your site from src or build, and you will still get Browserify’d files. SCRIPT_DEBUG will either run *.js or *.min.js, just like any other minified JS in core.

    This is a proposal

    I would like to hear feedback from the overall community and certainly from our fair share of JS-trained ninjas. A common reason to *not* do something like this is the barrier to entry for new developers. I would argue in this case that the code becomes MORE readable and understandable. I was shocked myself to see how much simpler it was to absorb one piece at a time once the code was laid out in modules.

     
    • deltafactory 10:07 pm on January 22, 2015 Permalink | Log in to Reply

      As someone who was trying to wrap my head around wp.media as recently as yesterday (with little success), I strongly support this from a development and maintenance perspective.

      • Primoz Cigler 6:12 pm on January 23, 2015 Permalink | Log in to Reply

        +1!

        I attempted several times to ‘load’ these enormous JS files to my brain in last few months. Every time with little to no success. As I am using RequireJS I would be more font of the later, but I absolutely agree with your foreseen troubles it would bring. So browserify + grunt it is, hurray!

    • Daniel Bachhuber 10:13 pm on January 22, 2015 Permalink | Log in to Reply

      I strongly support breaking apart our JavaScript. Browserify has become a new standard for the projects I’ve been working on.

    • Luis Rodrigues 10:22 pm on January 22, 2015 Permalink | Log in to Reply

      I for one applaud this initiative. There may be a couple of extra hoops to jump through, but they become virtually irrelevant with a properly configured Gruntfile to handle watching and building modules into the project.

      Big fan of Browserify, too. Browserify will handle AMD modules if you need them (using plugins like `deamdify`) and coexists with non-module dependencies much, much better than RequireJS. (Last time I got RequireJS and WP to work together, I had to mess with WP’s enqueue mechanism because the loader is rather sensitive to scripts that alter the DOM.)

    • K.Adam White 10:28 pm on January 22, 2015 Permalink | Log in to Reply

      Music to my ears, @wonderboymusic! Regarding the “barrier to entry” argument, in my experience the biggest barrier I’ve seen people hit while working with modules has been that in almost all circumstances, the complexity of setup and configuration required by a system like Require results in a front-loaded learning curve. Coming onto an existing project with a modular system is, by point of contrast, generally a pleasant experience because it is much more obvious where any particular piece of code lives.

      I’d throw in my $0.02 that since we’d need a build process anyway to ship production files, we should optimize for ease of development: whichever system requires the least work when adding a file. Webpack’s a good build tool that we’ve been using which supports both AMD and Node-style modules, as well.

    • Solinx 10:41 pm on January 22, 2015 Permalink | Log in to Reply

      Like K. Adam I found that working with modules does increase the initial barrier a bit for developers who rarely use javascript, but certainly not by much. And once the required code to work with modules is understood it makes things easier due to the clear structure and focussed content of each module.

      Currently we use AMD (require.js) and build our combined js file with Almond, but we’re strongly considering to switch to Browserify or Bower. AMD isn’t as widely supported by packages as we’d like, and Require.js appears to clash with our new general purpose build tool, Gulp.js.

    • Mike Schinkel 11:14 pm on January 22, 2015 Permalink | Log in to Reply

      I have nothing but support for this proposal. Great work!

    • faction23 1:09 am on January 23, 2015 Permalink | Log in to Reply

      Having the js broken up into modules will be great! Have you tested/considered your third option, the use of native es6 modules and then something like webpack with the 6to5 loader (es 6 to 5 transpiler)? es6 modules are finalized since last august – es6 was fully feature frozen then -and I’ve been working with 6to5 for a little bit and its rock solid. The main reason would be longer term viability. es6 modules are going to be here to stay guaranteed, plus they pretty well supersede common/amd modules…

    • Mike Schroder 1:41 am on January 23, 2015 Permalink | Log in to Reply

      This sounds most excellent.

      Thanks for putting this together!

    • Helen Hou-Sandi 4:06 pm on January 23, 2015 Permalink | Log in to Reply

      Really looking forward to some sanity in this area. Concerns I typically have about barrier to entry are mostly irrelevant here – components such as media already require a high level of understanding and some amount of experience, which makes familiarity with build tools as a requirement of development much more likely. As I’m understanding the proposal, build/watch won’t be needed if you don’t touch any involved JS.

    • Aaron Jorbin 8:42 pm on January 23, 2015 Permalink | Log in to Reply

      My concerns are the same as others, that this increases the barrier to entry. I think that as long as some documention get’s written for the core handbook that we can also reference and point to about how and why (with this post serving as a great foundation).

      One thing we will want to consider, if a file like wp-includes/js/media/models.js is going to be a generated file, we should exclude it from jshint.

    • Ben Doherty (Oomph, Inc) 12:42 am on January 25, 2015 Permalink | Log in to Reply

      You have my +1 in this effort. The media-* files are just too much to wrap one’s head around effectively, and in general we are in dire need of organization and better documentation for the entire Media Manager API so that developers can write better integrations. I don’t believe there will be much increase in barrier-to-entry by splitting these files up. As Helen said, developers who dig into this are likely to already be familiar with the build toolchain, but may not need even it when using the files for reference.

    • nikeo 3:04 pm on January 25, 2015 Permalink | Log in to Reply

      +1. Those types of workflows + tools have to be democratized. I don’t think there will be much increas in barrier-to-entry as well.
      Thanks!

    • lmartins 10:16 pm on January 25, 2015 Permalink | Log in to Reply

      Common JS with Browserify or similar every time. Any front-end person deals with them these days, so I imagine quite accessible to any dev.

    • Per Soderlind 11:13 am on January 28, 2015 Permalink | Log in to Reply

      + 1, understanding (your) wp.media code today is hard.

    • Andrew Ozz 6:07 pm on February 2, 2015 Permalink | Log in to Reply

      Don’t think this is going to make huge difference. Generally whether you edit few places in a single file or have to open and edit several files doesn’t make a difference. Your IDE should handle both cases transparently :) However agree that it is better to have a nice file structure that somewhat matches the JS structure.

      In my mind it is more important to improve/fix the “JS structure”, i.e. this:

      The main canvas on which these Views are stitched together are called Frames, which are themselves Views – tilting our use of Backbone more towards MVP, P standing for Presenter.

      We have Controllers, which are called States, but they belong to a Frame (Presenter! also a View!), so anyways…. for now….

      I’m hoping we will get that opportunity when implementing the image flow changes.

    • Dwain Maralack 9:25 pm on February 8, 2015 Permalink | Log in to Reply

      +1 for a more logical file structure. Most of the files here will most probably only be changed by experienced / core dev’s so the barrier to entry is not a major concern. The entry barrier can easily be solved by supporting documentation.

  • Eric Andrew Lewis 1:39 am on June 28, 2014 Permalink
    Tags: , media   

    Media Grid Update 

    Media Grid started as a standalone plugin by Shaun Andrews, which was a reimagining of the UI as an alternative to the traditional post list view in the Media Library. The argument was that images are the ubiquitous media type in most users’ libraries, so we should provide an interface to browse media visually.

    I joined the project in late April, attempting to integrate existing Media modal code. This work was merged into the standalone plugin, and into trunk(see #24716) in early June. In the process, I created documentation for the Media code, which is the most comprehensive resource for untangling the Backbone wires in media.

    Questions were raised about what problem the grid was solving, so in order to get a more hands-on understanding of user engagement with the Media Library, Jerry Bates performed user interviews. These confirmed our assumption that images are the pervasive media type, but also surfaced the fact that users identify media in different ways – some by the thumbnail, some by what post a media item is uploaded to, some by title.

    After a good amount of UX/UI chatter in weekly meetings, we decided we could serve users better by making a few changes to the original implementation merged into trunk. We’ve landed on mock-ups for a quick pivot, which I’m working on implementing . I’ll be dropping diffs for y’all Javascript Jedis to peruse on #24716, feedback welcome and appreciated. I hope to have merge-ables by Monday morning, and then to progress to user testing.

     
  • Eric Andrew Lewis 3:50 pm on May 29, 2014 Permalink
    Tags: media   

    Media component office hours 

    Weekly office hours (bug scrubbing and feature development chat) for the Media component will start tomorrow, combining with Media Grid’s weekly meeting.

    I’ve heard the Media component is “where tickets on trac go to die,” let’s see what we can do to thwart that status quo.

    Our Media Grid meetings have been held every Friday at 1700 UTC in #wordpress-ui. We’ll keep the same time, but move the meeting into #wordpress-dev.

     
  • Scott Taylor 5:43 pm on March 11, 2014 Permalink
    Tags: , media, ,   

    Audio/Video 2.0 Update – Media Modal 

    The latest major updates to Audio/Video before Beta were related to editing shortcodes in the medial modal. TinyMCE placeholders for audio and video have been added like so:

    00-placeholders

    When you click the placeholder, the media modal will now be launched. And when you get there, there are some new ways to manage HTML5 playback without writing anything by hand.

    01-audio-details

    Add Multiple Sources

    MediaElement.js provides cross-browser support for many extensions by providing Flash and Silverlight files that will bridge the browser support gap when necessary. Ideally, as many browsers as possible would be using native HTML5 playback. To ensure this, you need to specify multiple versions of your files. Since 3.6, the audio and video shortcodes have supported this by allowing you specify multiple extensions as attributes: mp3="yolo.mp3" ogg="yolo.ogg" wma="yolo.wma", etc.

    A quick and easy way to encode your files is to use FireFogg (works in Firefox only). Here are some instructions for audio and video files:

    HTML5 Audio

    • Click “Make web video”
    • Select a File (your iTunes files on a Mac are in: ~/Music/iTunes/iTunes Media/Music – Pick a tune!)
    • Click “Advanced Options”
    • Uncheck Video, make sure Audio is checked
    • Format: Ogg (Theora/Vorbis)
    • Set quality to 10.0
    • Optionally add metadata
    • Click “Encode” – make sure to change the extension in the file prompt from “.ogv” to “.ogg”

    HTML5 Video

    • Click “Advanced Options”
    • Make sure Audio and Video are both checked
    • Format: choose Ogg (Theora/Vorbis) or WebM (VP8/Vorbis)
    • Optionally add metadata
    • Click “Encode”
    • (Repeat these steps for the format you didn’t select this time)

    There is now a workflow to make adding these extra sources to shortcodes easy:

    02-audio-add-ogg

    Multiple sources are specified now. Make sure to click blue “Update” button to save your shortcode back to the editor.

    03-audio-sources

    Video Workflow

    Here is a video workflow, assuming your shortcode has no attributes and you click it:

    04-video-empty

    Add each video source:

    05-video-add-source

    Optionally add a poster image:

    06-video-sources

    Subtitles

    If you’re feeling CRAZY, add some Subtitles! Here’s a post about them. They’re pretty cool. Probably make sure your web server is serving .vtt files as text/vtt.

    07-webvtt

    Add your subtitles using our easy workflow:

    08-video-add-subtitles

    When you’ve added them, MediaElement knows to support them out of the box:

    09-video-click-subtitles

    Boom. Subtitles while your video plays:

    10-video-show-subtitles

    When you add your subtitles, if will show you a list of “track” elements you have added, you will still need to set your language manually – all will default to English. The idea is that you can add a track element per language. Tracks get stored as the body of your video shortcode.

    tracks

    Testing + Tweaks

    Now comes the time for testing and refining, while fixing esoteric bugs. PLEASE HELP! Put your UI + UX hats on, if you wear that kind of hat.

     
    • Scott Kingsley Clark 6:31 pm on March 11, 2014 Permalink | Log in to Reply

      Is Audio/Video modal stuff able to be integrated into separate fields from TinyMCE yet? Or too early to try testing that?

      • Scott Taylor 6:37 pm on March 11, 2014 Permalink | Log in to Reply

        Great question – yes:
        https://core.trac.wordpress.org/browser/trunk/src/wp-includes/js/tinymce/plugins/wpgallery/plugin.js#L95

        var frame = wp.media.video.edit( data );

        “data” is a shortcode.

        frame.on( 'close', function () {
        	frame.detach();
        } );
        frame.state( 'video-details' ).on( 
        'update replace add-source select-poster-image add-track', 
        function ( selection ) {
        	var shortcode = wp.media.video.shortcode( selection );
        	// shortcode is the updated shortcode, do something with it.
        	frame.detach();
        } );
        frame.open();
        
        • Manny Fleurmond 7:48 pm on March 11, 2014 Permalink | Log in to Reply

          Is there a version of the modal to be used outside of shortcodes and the editor, like how you can use the media select modal in plugins?

          • Scott Taylor 7:53 pm on March 11, 2014 Permalink | Log in to Reply

            I honestly hadn’t thought about other use cases until Scott’s comment. All of the code is in media-editor.js, media-models.js, and media-views.js which get loaded via `wp_enqueue_media()`. If those files + MediaElement are loaded, you can do whatever you want. The admin glue is the “wpgallery” TinyMCE plugin where we have shoved all of the shortcode handlers, playlists too. So, you need your own glue, but see my above comment, it’s pretty easy.

            • Manny Fleurmond 4:21 am on March 17, 2014 Permalink

              Is there a way to use the edit audio/video frames without shortcodes?

            • Scott Taylor 4:25 am on March 17, 2014 Permalink

              wp.media.audio.edit( ‘[ audio ]’ ).open();

            • Manny Fleurmond 4:39 am on March 17, 2014 Permalink

              So basically, I’d have to encode the data into shortcode before I could even use this modal, then.

            • Scott Taylor 4:47 am on March 17, 2014 Permalink

              wp.media({
              frame: 'audio',
              state: 'audio-details',
              metadata: {DATA_GOES_HERE}
              }).open()

            • Manny Fleurmond 4:58 am on March 17, 2014 Permalink

              Awesome and thank you. 2 more questions, then you can chase me away:

              1. What kind of metadata and how is it formatted? is it just a hash of audio type and url?
              2. Will the same events shown above ( ‘update replace add-source select-poster-image add-track’ ) work with this code?
              3. (wait, didn’t I ask for 3?) what does detach do?

              Thank you and sorry for all the questions. Digging through the code is tricky because I’m still new to backbone so I’m glad that people in the know such as yourself are will to share the knowledge. Can’t wait for documentation.

            • Scott Taylor 5:03 am on March 17, 2014 Permalink

              1. look at “defaults” here: https://core.trac.wordpress.org/browser/trunk/src/wp-includes/js/media-editor.js#L702

              2. Those events are in reaction to what happens in the media modal, they get set up here:
              https://core.trac.wordpress.org/browser/trunk/src/wp-includes/js/tinymce/plugins/wpgallery/plugin.js#L109

              3. Removes the frame from the DOM when the state is to be discarded or the modal is closed.

            • Manny Fleurmond 5:05 am on March 17, 2014 Permalink

              Thank you. I had just played around with the code in the js console and got an output from and update event. Thank you for the starting point. :)

    • Robert Chapin 7:47 pm on March 11, 2014 Permalink | Log in to Reply

      I love that I can upload audio files and have my own player without embedding cross-domain scripts. This is awesome.

    • Gabriel Koen 11:57 pm on March 12, 2014 Permalink | Log in to Reply

      This stuff is awesome!

    • Looimaster 11:09 am on March 18, 2014 Permalink | Log in to Reply

      @Scott Taylor: Would it be possible that you update Codex pages with examples for developers such as `wp.media.audio.edit( ‘[ audio ]‘ ).open();` etc.? It would be beyond awesome to have this documented in detail, with properties, methods, exact parameters, return values, examples etc. I’m not that proficient in JavaScript and it’s hard to find information such as what the actual data (parameters) are for `wp.media()` or for `metadata: {DATA_GOES_HERE}`. If you hadn’t told us that we can do `wp.media.audio.edit( ‘shortcode-here’ ).open();` then few people would be able to get to know that from the source code.

      I think that this page https://codex.wordpress.org/Media_Library could be made like this: https://codex.wordpress.org/Embeds or this: https://codex.wordpress.org/Child_Themes It’s super informative and it’s the only source of information that I’m able to understand quickly and use in plugins and themes.

      PS
      This new media library is an AWESOME step forward. You’re great!

    • Stephanie Leary 3:11 pm on March 23, 2014 Permalink | Log in to Reply

      Where is the relationship between video and subtitle file stored? I’ve played with the latest nightly and I can’t find anything in the posts, postmeta, or options tables that indicates which subtitle file goes with which video. I’d like to be able to query videos with subtitles vs. those without.

    • manuelmasia 4:40 pm on April 19, 2014 Permalink | Log in to Reply

      Is there any way to filter the box to add some fields: besides “Autoplay” and “Loop”? I would like to add the ability to select the video start volume and another field to define whether to use the video as background (according with a theme I’m working on). Is there any tutorial or resources? TIA, Manuel :-)

  • Scott Taylor 9:04 pm on February 20, 2014 Permalink
    Tags: , media, playlists,   

    Audio / Video 2.0 Update – Playlists 

    Previously: https://make.wordpress.org/core/2014/01/29/audiovideo-2-0-update/

    It has been a slow and steady burn of commits related to media and necessary for playlists:
    [27059], [27063], [27907], [27100], [27127], [27209], [27212], [27213], [27214], [27215]

    A few items for conversation….

    Thumbnails for Audio and Video

    We have been parsing the ID3 tags in audio and video files on upload since 3.6. ID3 tags, in many cases, contain the binary bits for an image related to the item (album cover, still, etc). On upload, if your theme and the relevant pseudo post type for attachment:audio or attachment:video registered support for thumbnails AND your current theme supported it, we would upload the image bits and attach to the media item as its featured image.

    So basically it was a hidden feature, because it required you to jump through hoops to turn it on.

    add_post_type_support( 'attachment:audio', 'thumbnail' );
    add_post_type_support( 'attachment:video', 'thumbnail' );
    add_theme_support( 'post-thumbnails', array( 'post', 'attachment:audio', 'attachment:video' ) );
    

    On top of that, if you switch themes, and the theme doesn’t support thumbnails for audio or video, the images will no longer appear alongside the media on the Edit Media page. Weird.

    Playlists are best enjoyed with images, videos are best enjoyed with poster images. Soundcloud is doing some awesome things with images in their embeds – see a few on the homepage here: http://highforthis.com. Moral of the story: I think this support should be on by default. Alternately, we could add that code to every default theme’s functions.php, but then what if you switch themes…

    Playlist UI

    As I mentioned previously, the UI for playlists needs to be:
    1) Adaptable
    2) Extensible
    3) Generic

    Translation: it needs to show up in a theme without much drama, inherit the style of the theme, respond the theme’s $content_width, all while allowing you to completely override this behavior. So, what I have is an ultra generic design controlled by settings in the media modal:

    Playlist Settings
    I have tested this in the last 5 default themes:

    Twenty Fourteen

    Screen Shot 2014-02-20 at 3.47.59 PM

    Twenty Thirteen

    Screen Shot 2014-02-20 at 3.48.58 PM

    Twenty Twelve

    Screen Shot 2014-02-20 at 3.49.22 PM

    Twenty Eleven

    Screen Shot 2014-02-20 at 3.50.10 PM

    Twenty Ten

    Screen Shot 2014-02-20 at 3.50.49 PM
    I would like to drop this code in soon, but I wanted to give an opportunity for feedback. All of this can easily be iterated upon once it goes in.

    Documentation of 3.5 Media Code

    This is ongoing – there has been a lot of code churn in the Backbone code, by myself and others, I’ll be picking this back up once that settles down.

     
    • Chris Reynolds 9:16 pm on February 20, 2014 Permalink | Log in to Reply

      I’m soooo excited about this feature.

    • Eric 9:20 pm on February 20, 2014 Permalink | Log in to Reply

      Oh sweet! Please oh please tell me that API support is planned!

    • ScreenfeedFr 9:50 pm on February 20, 2014 Permalink | Log in to Reply

      I’ve been working on similar things recently (bulk import of audio files + multiple playlists among other funny things).

      I came across a problem for the thumbnail (I use the ID3 tag too): detect duplicates. If you upload 10 audio files from the same album, you’ll have 10 identical image attachments too 😐

      So far, the only way I have to avoid creating the same image attachment multiple times, is to store the raw data in an array until the bulk import ends, and check for duplicate for each audio file. So it’s pretty limited, but it works, as long as you import all the album files within the same import.

      Did you find a way?

      • Manny Fleurmond 3:57 am on February 21, 2014 Permalink | Log in to Reply

        Maybe store the images on a per album basis?

        • Manuel Schmalstieg 10:12 am on February 21, 2014 Permalink | Log in to Reply

          @Manny : This would be a problem for Nine Inch Nails though, where some albums have different artwork for each track.

        • ScreenfeedFr 12:35 am on February 23, 2014 Permalink | Log in to Reply

          @Manny @Manuel Schmalstieg:
          Indeed, each track could have its own artwork within the same album, that’s why I ran into this duplicate detection :/
          It’s too bad but I think we can’t do anything for that :(

          • Scott Taylor 1:22 pm on February 24, 2014 Permalink | Log in to Reply

            we could store an md5 of the bits as the guid and possibly check before uploading – might tackle that after this initially goes in.

            • ScreenfeedFr 9:21 pm on February 24, 2014 Permalink

              That sounds like a good solution to me :)
              I’m looking forward to see what will happen with themes/plugins that use the guid as url 😀 (evil laugh)

    • bfintal 1:28 am on February 21, 2014 Permalink | Log in to Reply

      Playlists look geat!

    • Justin Kopepasah 8:38 am on February 21, 2014 Permalink | Log in to Reply

      This feature looks awesome. Great work Scott. Looking forward to diving in.

    • Manuel Schmalstieg 10:36 am on February 21, 2014 Permalink | Log in to Reply

      Looking awesome indeed! Just one little thing: on a site that is run by a band, they should be able to switch off the “by NAME OF THE ARTIST” after each track title. This seems even more important than showing/hiding the track numbers. Especially if it’s an annoyingly long band name, all in uppercase. :)

    • Diego de Oliveira 5:35 am on February 23, 2014 Permalink | Log in to Reply

      I would love to see an easy way to get thumbnails for uploaded audio / video! Working on the CEUX project we have content blocks for media. Getting a poster image for fetched video (at least for youtube and vimeo) is easy, as the providers have an API that allows it, but getting this for self hosted media is another case. And media playlists is a great feature that we didn’t thought about yet. Maybe we’ll need a content block for that!

    • tiaurus 1:45 pm on March 28, 2014 Permalink | Log in to Reply

      What about Cyrillic and other non-latin symbols in audio tags in playlist?

    • sourceforge 10:49 am on April 17, 2014 Permalink | Log in to Reply

      The metadata is extracted from mp3 uploaded, what-if I am using remote url, can we fetch the ID3 tags? or maybe have global and local options for a playlist that can edited manually using shortcode

    • sourceforge 11:05 am on April 17, 2014 Permalink | Log in to Reply

      [26825], sorry

    • jk3us 7:24 pm on April 17, 2014 Permalink | Log in to Reply

      Can you have a playlist of externally hosted audio files?

    • cebab-pl 10:06 am on May 9, 2014 Permalink | Log in to Reply

      Hi Scott!
      How can i add thumbnail video (may by icon post for video) for a playlist video?
      I don’t want title files, but i want image (poster)

  • Scott Taylor 8:23 pm on January 29, 2014 Permalink
    Tags: media   

    Audio/Video 2.0 Update 

    MediaElement

    Upgraded to 2.13.2: [27059]

    Documentation of 3.5 Media code

    #26870 Add (JSDoc)umentation to the Backbone-centric Media files

    This is underway, with several commits already. Slow-and-steady approach. Have started with annotations, gradually adding top-level comments to all class methods. I have been using pseudo-code to map out each file: https://gist.github.com/staylor/10115a0f455e16c6eafd.

    Start with identifying every piece of the tree, then making sure all methods have each piece of the documentation.

    Dashicons

    #26650 Replace media file type icons with Dashicons

    These will replace the “crystal” icon set. For AV2, we are interested in replacing the icons that show up in the media list tables, and using the icons in the placeholders for Audio, Video, Playlist, and Video Playlist shortcodes in TinyMCE content.

    wpautop() changes

    #26864 Consolidate handling of object, audio and video tags in wpautop

    PHP and JS versions of wpautop() need to respect the inner elements of media HTML tags for media: <param>, <embed>, <track>, <source>
    Initial patch – ongoing work from azaozz

    #26628 Use the content of a video shortcode when provided – depends on ^

    Placeholders (TinyMCE views) for Audio / Video shortcodes

    media

    Depends on: #26628

    Chromeless YouTube via ME.js

    #24764 Support mediaelement.js YouTube sources in the video shortcode

    This has a patch: https://core.trac.wordpress.org/attachment/ticket/24764/24764.diff

    I have just been letting it soak in the wild for a day.

    Metadata Regeneration for audio/video

    #26825 Add ability to generate metadata for audio / video files on-demand

    This currently has 2 implementations.

    1. a button on the Edit Media page if your audio or video does not have a row in postmeta for _wp_attachment_metadata (! isset(), not ! empty())
    2. On the fly in wp_prepare_attachment_for_js()

    #1 requires the user to care

    #2 IMO does not scale, as the media modal loads ALL of your media in one fell swoop upon being opened – let’s say you have 300 videos uploaded pre-3.6: there will be no metadata, so you might melt your filesystem by scanning and analyzing 300 files at once.

    This is still in the exploratory phase – would love some UX thinking on this.

    Playlist and Video Playlist shortcodes

    #26631 Add a “playlist” shortcode

    The TinyMCE view code can probably be merged into the wpaudiovideo TinyMCE plugin once #26628 goes in, which has its own dependencies. As I mentioned, Playlists and Video Playlists probably need their own identifying icons.

    I have created a basic UI for playlists that inherits styles from your existing theme. The HTML and CSS is minimal and generic. Here are some screenshots of it in action:

    Twenty Ten

    Screen Shot 2014-01-29 at 3.06.07 PM

    Twenty Eleven

    Screen Shot 2014-01-29 at 3.05.12 PM

    Twenty Twelve

    Screen Shot 2014-01-29 at 3.04.45 PM

    Twenty Thirteen

    Screen Shot 2014-01-29 at 3.05.52 PM

    Twenty Fourteen

    Screen Shot 2014-01-29 at 3.05.31 PM

     
    • @ubernaut 8:31 pm on January 29, 2014 Permalink | Log in to Reply

      lookin’ good!

    • Andrew Nacin 8:31 pm on January 29, 2014 Permalink | Log in to Reply

      Tagging this ‘media’ so it shows up on https://make.wordpress.org/core/components/media/. (Oh hey look at that…)

    • Jose 11:05 am on January 31, 2014 Permalink | Log in to Reply

      That looks so cool. Can’t wait to play with it when I get home.

    • David Lingren 7:59 pm on February 21, 2014 Permalink | Log in to Reply

      First, let me applaud Scott’s efforts in “Documentation of 3.5 Media code”. The comment log in Ticket #26870 shows the complexity of this vital task!

      As a plugin author struggling to add functionality to the Media Manager Modal Window I’ve spent countless hours going over the WP code and the code of other plugins which go to conflicting extremes to extend the WP code. JSDoc comments are much needed and appreciated, but don’t provide much insight to the larger design concepts behind the major parts of the work, such as controller/StateMachine/State/states/workflow and Library/toolbar/menu/router/Region/Selection.

      I’ve seen (very popular) plugins that copy source code from media-views.js, modify it and then replace the WP method altogether by assigning their function to the WP .prototype. There must be a better way…

      Are there any efforts or plans to illuminate the thinking behind Media Manager and/or to make it possible for plugins to cooperate on extending it?

      Thanks again for all your hard work on this difficult and important task!

  • Scott Taylor 6:47 pm on January 13, 2014 Permalink
    Tags: , , , media, , Underscore,   

    Audio / Video 2.0 – codename “Disco Fries” 

    Some history:

    I wanted to do a Make post on my wants for Audio / Video in 3.9 to solicit feedback and spark some discussion about what the community wants / needs / doesn’t want / doesn’t need. Adding audio / video in 3.6 was a great first step, but there are some things we can do to continue to modernize Media and give our huge user base even more ways to display and manage their content. We can also make some changes that help developers navigate the new world of MediaElement.js, Backbone, and Underscore.

    First Things First: New Icons

    #26650 Replace media file type icons with Dashicons

    There are some lingering icons in the admin that don’t look as pretty as their MP6ify’d brethren

    Document the “new” Media code introduced in 3.5

    In 3.5, we overhauled Media. @Koop produced some beautiful code, and a LOT of it. Raise your hand if you’ve ever dived in and tried to program against it? Raise you hand if you understand how all of it works? Me neither. As a community, we need to help each other learn what it is and what it does. Documentation will go a long way in getting us all on the same page. Do we have a documentation standard for JS? We need one. While this isn’t the easiest place to start, it is a necessary one. I would be happy to spend time on this, as I have spent many hours recently reading the code and learning how it works. The main files: media-editor.js, media-views.js, media-models.js

    Support subtitles for Video

    #26628 Use the content of a video shortcode when provided.

    This ticket speaks for itself, and already has a patch.

    Generate audio/video metadata on-demand

    #26825 Add ability to generate metadata for audio / video files on-demand

    Add “Playlist” and “Video Playlist” shortcodes

    #26631 Add a “playlist” shortcode

    Adding inline players for audio and video was a great first step. How do I add music to my site? Just upload an MP3 and drop the URL on a line by itself. Done. Or use the audio shortcode. This works most of the time, but can be a little clunky if you want to share an album of your tunes. MediaElement doesn’t “support” playlists out of the box, but MediaElement is JavaScript, and with JavaScript and little UI elbow grease, we can EASILY support playlists.

    My ticket already contains a patch, but is still considered a work in progress. I think the playlist shortcode should produce markup that does the following:

    • Works out of the box with any existing theme: the HTML should be semi-bulletproof. Many of the Player libraries make heavy use of DIVs instead of items that might be overridden easily with CSS: LIs and the like.
    • Gives the developer total control if they want to write their own implementation
    • Exposes enough data to the page so the themer/dev can make their own decision regarding display of album cover, track meta, captions, etc.

    My current implementation drops data onto the page for each playlist inline. A wrapper div “.wp-playlist” will have a script tag in it with type=”application/json”. I do this so that if ‘wp-playlist.js’ is unenqueue’d, the developer still has the data necessary to write their own implementation. The data is reachable in JS like so:

    var data = $.parseJSON( el.find('script').html() );
    

    My current UI for playlist is a basic one, and uses Backbone Views to render the tracklist on load and update the “current” playing track view. There are 2 camps of people when it comes to “JS on the frontend” – one who doesn’t like it (others) and one who says “who cares” (me). One of the reasons I am posting this at the beginning is so we can flesh out issues like this.

    Abstract Gallery logic into “Collection” logic that Galleries, Playlists, etc can use with minimal registration code

    I have already done a first pass at this in the playlist shortcode patch. It goes like this: a “gallery” is really a “collection” of attachments of type “image.” A “playlist” is really a “collection” of attachments of type “audio.” So they should extend or be instances of a “collection”-type class. Currently, the Gallery code has to be dupe’d. By abstracting this code, Gallery, Playlist, Video Playlist, + any other “collection” of media type can be registered minimally.

    Other Ideas

    • In our playlist JS code, emit events that others can hook into – maybe a video playlist is: News clip, ad, news clip, ad, etc. When emitting events before / after an ad, the dev could disable next/prev buttons
    • Make a playlist embeddable on other sites via an iframe or embedded markup
    • Register an endpoint for audio / video that will expose the “embed code” via oEmbed

    Thoughts?

     
    • Joe Dolson 7:00 pm on January 13, 2014 Permalink | Log in to Reply

      Definite thumbs up from the Accessibility team for adding captions support. In addition to that, I’d like to suggest making it possible to enable keyboard accessibility more easily than it is right now. MediaElement.js includes settings which enable this — one is enabled by default, which enables keyboard access to the controls, but without ‘alwaysShowControls’ enabled, it’s not possible for a keyboard dependent user to move focus onto the player, so they can’t take advantage of any of those controls.

      I’d like to see a localization variable so that the MediaElement settings can be adjusted via wp_localize_script, but for keyboard accessibility it may be valuable to make this an option, either via shortcode or settings, so it’s possible for non-programmers to enable keyboard accessibility.

      I’ve got a plug-in, Accessible Video Library that hacks in support for alwaysShowControls by deregistering the default wp-mediaelement script, but that’s not an ideal solution.

      It would also be nice if the caption selector within MediaElementjs could be made keyboard accessible; though that’s probably something that should be handled as a patch to MediaElementjs itself.

      The Add Media Panel, in addition to new documentation, is in desperate need of accessibility work. See #23560. And several other tickets; there are quite a few accessibility tickets on the Add Media Panel.

    • Aaron Jorbin 7:03 pm on January 13, 2014 Permalink | Log in to Reply

      RE: JavaScript Docs.

      the JSDoc standard is pretty close to the phpdoc standard that we use for PHP and thus it makes the most sense in my eyes. There was a passing IRC conversation between members of the team that did the jshint work and the inline documentation that a future project could focus on improving the documentation of our JS. The media files seem like a great place to prototype this.

      http://usejsdoc.org/ is the standard.

      • Eric Andrew Lewis 8:02 pm on January 13, 2014 Permalink | Log in to Reply

        Yes to inline docs. However, I think we’re going to need an alternate form of documentation completely separate from inline to give an easily understandable overview of how media works.

        • Sam Sidler 8:37 pm on January 13, 2014 Permalink | Log in to Reply

          What did you have in mind?

          The inline docs will soon (ish) be present on developer.wordpress.org, which will help the visibility of them. Do you mean something more like a walkthrough on exactly how it works? I’m trying to think of where the best place for that would be. Definitely on the developer hub, but probably not part of the theme/plugin handbooks.

          • Eric Andrew Lewis 10:24 pm on January 13, 2014 Permalink | Log in to Reply

            Not sure to be honest. I think there are limits to inline documentation. Inline is always so contextual to the code it surrounds; separate documentation can speak of overarching design principles and secondary information that doesn’t pertain to any code block in particular.

            WP-API’s documentation for example is quite verbose, and covers a range of topics including the API’s philosophy, tutorials, schema details, etc. None of this could fit easily into inline documentation.

            Full disclosure: I’ve only used the media library cursorily, and am not even sure what secondary documentation for it might look like, so I may be off-base here.

            • Ryan McCue 1:11 pm on January 16, 2014 Permalink

              This is one of the benefits of the new developer site being WP-based: we can mix in inline docs with full pages. (It’s also one of the reasons WP-Parser is architectured that way.)

          • Gregory Cornelius 6:01 pm on January 15, 2014 Permalink | Log in to Reply

            Documenting the parts of the WordPress Backbone code that form more of a “public” API that is intended to be re-used in plugins with a few examples makes sense.

        • Aaron Jorbin 8:45 pm on January 13, 2014 Permalink | Log in to Reply

          I agree that some good tutorials and foundation docs are important, but I think that inline docs can help move us forward.

    • two7s_clash 7:17 pm on January 13, 2014 Permalink | Log in to Reply

      I wanted to point out that for a few years the flash player did support playlists: https://plugins.trac.wordpress.org/ticket/1869/ See also: http://jamesfishwick.com/3-6-audio-so-what-happens-to-blogs-using-the-old-jetpack-shortcode-for-generating-playlists/ Furthermore, I had a popular plug-in that did many of these things you list here called Jetpack Easy Playlists: http://jamesfishwick.com/software/jetpack-easy-playlists/ It worked on your “gallery” paradigm. The playlist still works if you hack Jetpack’s shortcode module and disable its check. If you want this to work with 3.6 and up, edit this file: /wp-content/plugins/jetpack/modules/shortcodes.php

      Change line 51 to: “if ( version_compare( $wp_version, ’3.9-z′, ‘>=’ ) && stristr( $include, ‘audio.php’ ) )”

    • @ubernaut 7:58 pm on January 13, 2014 Permalink | Log in to Reply

      i think this all sounds awesome.

    • s3w47m88 8:02 pm on January 13, 2014 Permalink | Log in to Reply

      We provide custom Theme WordPress websites to our customers and the primary challenge we face, regarding media, is that customers frequently are embedding video from sources beside the big guys (YouTube, Vimeo, etc…) so they have HTML or something. But that requires them to have access to the HTML (insert blood curdling scream here). What would be a potential solution is a simple widget on the right of Pages/Posts that allows them to paste whatever code they have, then see a rendered version without reloading the page, then the ability to drag that rendered video sample into the Visual Editor.

      As much as I like shortcode, customers still don’t get it for some reason. Anything they hear the word “code” they freak.

      • Sam Sidler 9:16 pm on January 13, 2014 Permalink | Log in to Reply

        What you’re talking about sounds a lot like the CEUX project (currently on hiatus).

      • Scott Taylor 9:18 pm on January 13, 2014 Permalink | Log in to Reply

        the shortcodes get inserted automatically from the Media Library, don’t have to be written by hand – they are just the easiest way to save the most minimal data necessary in the post content without hardcoding markup or URLs

    • Tom Auger 9:31 pm on January 13, 2014 Permalink | Log in to Reply

      Sounds like a pretty wide scope of things. I’m primarily interested in extending the Media Editor, and have prepared a feature-as-plugin to this effect as a proof-of-concept. As I missed the deadline for 3.9, I was planning on waiting until the next round to bring this forward, but if work is being done in media-editor.js and friends, perhaps this is a good time to look at the challenges that the current architecture of the Media Editor presents, and my proposed solutions to it.

      • Manny Fleurmond 4:35 am on January 14, 2014 Permalink | Log in to Reply

        I was planning on tackling the media editor as well, specifically making it more modular and using Backbone. Would love to see what you have so far

        • Tom Auger 3:18 pm on January 14, 2014 Permalink | Log in to Reply

          Okay, well, when I get a few hours, I’ll throw up a post and share the code. I’ve completely modularized the “editor groups” – which are basically the Media Editor’s version of Meta Boxes. It is currently **possible** to extend the media editor without core modifications, but it’s really ugly and involves duplicating some core files. This is one of those cases where it would really benefit from some well-placed hooks.

    • mttktz 2:44 am on January 14, 2014 Permalink | Log in to Reply

      Half of what’s important in WP is making sure that it is easy to create great stuff and publish it on the web.

      The other half is people finding that great stuff. Right now other platforms do a better job of sharing and redistributing great stuff. Better or more sophisticated oembed would probably mean more WP posts that look “right” when someone shares them on another site or tries to link to them.

      That sounds really interesting to me – but I’m not sure what you are thinking about there.

    • David Lingren 3:48 am on January 15, 2014 Permalink | Log in to Reply

      I was very excited to read this post!

      As the author of Media Library Assistant (https://wordpress.org/plugins/media-library-assistant/), I have devoted quite a bit of time and effort to enhance the Media experience in WordPress. One of the frequent comments I get in my support forum is “this should be in core”, and I would be delighted to see any relevant parts of my work get to a wider audience.

      Two of your proposals are of particular interest. First, “Document the “new” Media code introduced in 3.5″. I have had several requests to enhance the Media Manager Modal Window, and I have devoted a frustrating amount of time to understanding the code behind it. One of the best aspects of WordPress is its provision for actions, filters and other extension mechanisms; these are nowhere to be found in the Media Manager code. Consider these support requests I’ve received and responded to:

      Media sorting (https://wordpress.org/support/topic/media-sorting)

      MLA and ACF image upload conflict (https://wordpress.org/support/topic/mla-and-acf-image-upload-conflict)

      category selection when uploading media (https://wordpress.org/support/topic/category-selection-when-uploading-media)

      Hide edit cat and tags in media manager (https://wordpress.org/support/topic/hide-edit-cat-and-tags-in-media-manager)

      Different form field for category selection? (https://wordpress.org/support/topic/different-form-field-for-category-selection)

      Problem with Simplefields (https://wordpress.org/support/topic/problem-with-simplefields)

      filter not showing in modal popup window for image (widget https://wordpress.org/support/topic/filter-not-showing-in-modal-popup-window-for-image-widget)

      I would welcome an opportunity not just to understand, but to improve the Media Manager code.

      Second, you propose to “Abstract Gallery logic into “Collection” logic that Galleries, Playlists, etc. can use with minimal registration code”. This is a great idea, and should be extended to all of the items that can be stored in the “Media” Library, not just image, audio and video items. There are many sites using WordPress to manage large collections of other assets such as PDF documents. Let’s give them some attention as well.

      I look forward to following your progress and finding a way to contribute something to it. Thank you!

    • spacedmonkey 4:31 pm on January 19, 2014 Permalink | Log in to Reply

      With this move away from galleries to collections, (an idea which I really like btw), the gallery code should be rewritten to use walkers. This would make it much much easier to override the markup that galleries output.

    • Gregory Karpinsky 9:16 pm on January 22, 2014 Permalink | Log in to Reply

      Mechanism of inserting video from a YouTube channel: show a list of available videos, and let choose one?

      Can be based on formatting the output of a JSON like this one:
      https://gdata.youtube.com/feeds/api/users/musicinsummer/uploads?max-results=25&alt=json&orderby=published&format=5

      Same can be used to show all recent uploads to a channel.

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel
Skip to toolbar