WordPress.org

Ready to get started?Download WordPress

Make WordPress Core

Updates from May, 2012 Toggle Comment Threads | Keyboard Shortcuts

  • Helen Hou-Sandi 7:16 pm on May 28, 2014 Permalink | Log in to leave a Comment
    Tags: , ,   

    Summary for 5/21, Agenda for 5/28 

    Summary of 5/21 dev chat (IRC log):

    General

    • Posts like the i18n goals one serve as a great model for anybody who has ideas for a feature or component roadmap, whether that’s within one cycle or longer term: a list of concrete goals that can be divided up into specific tasks.
    • The make/* blogs should be used as much as possible for discussions and progress updates. Teams that have been using separate P2s but should consider using the make.wordpress.org instead for wider reach and more active participation from the community.
    • Keep in mind that plugins are supposed to encourage rapid and possibly wild experimentation – please do not discourage that.
    • Think of meetings as blocked out times where you can more reliably get a group together and get unstuck as needed, but we should take advantage of async (Trac, P2) and adhoc (IRC outside of meeting) discussion as much as possible.

    Team Updates

    • i18n goals for 4.0 have been posted, @nacin is seeking people to help with a lot of it. @yoavf, @kovshenin, @iandunn, @coffee2code, and @otto42 have done or will do some of the i18n tasks.
    • JSON REST API was given another week to collate a detailed compare-and-contrast with the other available APIs, including the JSON API plugin and Jetpack/.com’s API, and proven client implementations.
    • Media Grid has a narrowed scope for 4.0 inclusion: something more visually driven than the standard list table, much like theme browser brought to themes and is being investigated for plugins in 4.0. Will also fix some long-standing issues that were brought in with 3.5.
    • Editor improvement ideas: @markjaquith and @avryl have put together a proof-of-concept plugin that we should smooth out and make a decision on.

    Agenda for 5/28:

    • Make final decision on JSON REST API.
    • One sentence updates from various groups – i18n, media grid, plugins, oEmbed, etc. Come prepared.

    Please propose any other agenda items in the comments below.

     
  • Alex Shiels 10:56 am on May 28, 2014 Permalink | Log in to leave a Comment
    Tags:   

    Improving the Plugins page 

    The WordPress Plugins page has barely changed in 5 years or more – compare WP 2.7.1 with 3.9.1.

    The very first page seen by a new user who clicks on the Plugins tab is a list view showing two installed plugins. The main thing thing that has changed since 2.7 is that the way to find and install new plugins has become less obvious.

    Similarly, the plugin-install page has barely changed in that time: WP 2.7.1 and 3.9.1.

    The default page is very much geared towards maintenance by established users. The most common interaction is probably deactivating and reactivating plugins for troubleshooting – certainly a necessary task, but I think it misses a good opportunity for helping people to find and use the plugins they need.

    There are a number of improvements that could be made with relatively minor changes:

    Improve the experience for new and infrequent users.

    • The obvious fix here would be to make the path for discovering and installing new plugins much more obvious than the “Add New” link. Perhaps even go as far as making plugin-install.php the default page.
    • The Search Installed Plugins box on plugins.php is easily mistaken for a plugin directory search. Either make it less confusing, or use a unified search.
    • When searching for plugins in the directory via plugin-install.php, tailor the results to the current WP version. Give more weight to those that are compatible with the current version, and/or filter out those that are likely to be incompatible.

    Help users to discover the plugins they need, especially the most robust and well-maintained.

    • On the Add New page, the most common tags in the cloud are “widget”, “post” and “plugin”. It’s next to useless. Replace it with a well-defined list of categories more in line with common needs: “contact forms”, “image galleries”, “security” and so on.
    • Improve the quality of plugin directory search results generally. Incorporate things like ratings, support stats, age, usage stats, and update frequency in the relevancy scores.
    • Augment or replace version compatibility votes with stats based on active installs per WP version.
    • Re-evaluate the tabs on the Install Plugins page. Is “Newest” helpful? Should “Popular” or “Featured” have a summary on the main page?
    • Improve the algorithm used for averaging ratings, to smooth out errors for plugins with only a handful of ratings.
    • Change the columns displayed in Search Results. “Version” doesn’t need a column; but compatibility and age ought to be shown.
    • Also show compatibility for installed plugins #27699
    • Improve the ordering and filtering possible in the plugin search API #12696 and #27316

    Improve the way detailed information is given about plugins.

    • Either eliminate the thickbox for the plugin details, or make it more consistent with the theme browser (allow next/prev)
    • Add a Details view for installed plugins #17902
    • Show reviews in the detailed view #22599
    • Show contributors in the detailed view #19784
    • Show the plugin’s banner in the detailed view, and generally make it more consistent with what’s on the web site.

    Help and encourage developers to publish and maintain their plugins.

    • Support screenshots, logos, or banners in the search results, installed plugin list and plugin directory.
    • Do a better job of handling ratings, reviews, updates, and support stats, especially when determining search ordering and popularity.
    • Improve the profile page to list version compatibility, support stats, and other useful info for all your plugins.
    • Add a version requirement check and/or upgrade prompt #26909 and #27323

    And finally there are some other tickets suggesting improvements and fixes that could use a second look:

    • #28085 – Recently Updated plugins view (recently updated installed plugins)
    • #20578 – allow delete without uninstall
    • #27110 – allow filtering the plugin list
    • #26202 – bugfix for thickbox title truncation
    • #27623 – search results for a single space
    • #27994 – handling of automatic plugin deactivation in the event of an error

    I’m working on the API side, starting with improvements to search quality. There are tickets above for many of these items already. If you’d like to help out, keep an eye on the Plugins Component in trac, open and help with tickets. Or leave a comment here with your suggestions if you’re interested.

     

     

     

     

     

     
    • chriscct7 11:12 am on May 28, 2014 Permalink | Log in to Reply

      One thing that would also be really cool is if you could do like an ecommerce style cart in the plugin area. So an example would be I search for SEO plugins, find one I like and “add to basket”. Search for caching plugin, find one I like, add to basket. Search for cat shortcode plugins, find two I like in the results, add both to basket. Then at the end, I click an “install all plugins in basket”.
      This would solve an issue I have with the plugins area which is it takes forever to install more than say 4 WordPress plugins, because you have to either have a 4 tabs of plugin-install open to do them simultaneously, or go back to back to back to back which takes forever. Just a thought.

    • Ajay 11:17 am on May 28, 2014 Permalink | Log in to Reply

      I don’t think making the “Add New” page as the default is a good option. You’re more likely to visit the plugin page to view / disable / access links of the existing plugins rather than install new plugins.

      It would be good to have a single column with rating, no. of downloads, age, etc. rather than separate columns in order to give more width to the Description section.

      • chriscct7 11:19 am on May 28, 2014 Permalink | Log in to Reply

        I agree with Ajay. I don’t think Add New as default is a good idea

      • Brad Touesnard 12:02 pm on May 28, 2014 Permalink | Log in to Reply

        +1

      • Chip Bennett 3:33 pm on May 28, 2014 Permalink | Log in to Reply

        +1. The default Plugins page should be installed/active Plugins.

        If anything, the default Plugins page should aim toward facilitating Plugin management. Things I would like to see:

        1. Plugin Settings page link added by default
        2. Plugin categorization

        • Torsten Landsiedel 3:09 pm on May 30, 2014 Permalink | Log in to Reply

          +1 for default to installed plugins (to be consistent to other pages UI)

          +1 for adding the settings link by default

          and another +1 for categories for plugins :)

        • Torsten Landsiedel 3:23 pm on May 30, 2014 Permalink | Log in to Reply

          Speaking of plugins: What about making it possible to connect the support tab of a plugin with the international forums like xx.forums.wordpress.org instead of wordpress.org/support to make local/translated support possible. Or better: Make the whole plugin page multilanguage. This would be an huge enhancement for the plugin page in WP too.

      • michalzuber 4:21 am on May 29, 2014 Permalink | Log in to Reply

        +1

      • daveshine (David Decker) 3:01 pm on May 29, 2014 Permalink | Log in to Reply

        +1

      • The Portland Company 6:59 pm on May 29, 2014 Permalink | Log in to Reply

        That’s subjective. As a developer; sometimes I am deactivating, other times I’m installing. My customers are usually installing. And I’m sure there are other people groups with different applications as well.

        The most ambiguous model would seem to be an Option in the Screen Options (or Settings, whatever) that allows the User to configure the default page to their liking. Then, apart from that, it will remember what section/tab they were on so when they navigate away and then back again they can continue.

    • camu 11:23 am on May 28, 2014 Permalink | Log in to Reply

      Two words: plugin dependencies :)

      https://core.trac.wordpress.org/ticket/11308

    • Jacob N. Breetvelt 12:12 pm on May 28, 2014 Permalink | Log in to Reply

      I would like to add a feature request: the possibility of re-install without delete, i.e. forced update with the same version no.

      • crzyhrse 1:55 pm on May 28, 2014 Permalink | Log in to Reply

        +1

      • Ipstenu (Mika Epstein) 5:05 pm on May 28, 2014 Permalink | Log in to Reply

        https://wordpress.org/plugins/baw-force-plugin-updates/ can do that so it SHOULD be addable to core.

      • Dovy Paukstys 8:23 pm on May 28, 2014 Permalink | Log in to Reply

        +1

      • David Lingren 10:42 pm on May 28, 2014 Permalink | Log in to Reply

        Great idea. I would also support “forced update with the current stable version”. I have had a few occasions (my fault, of course) when I posted a new version, found a bug and reverted to the previous version. There are always a few folks who got the new version before I could recall it, and they are stuck with the higher version number.

        In addition, it might be useful to have a “go back to the previous version” option when an update causes a problem or just isn’t wanted.

        I realize both of these can be complicated by database changes, etc. but they are worth considering.

        • The Portland Company 7:03 pm on May 29, 2014 Permalink | Log in to Reply

          +1

          Furthermore we really need to require developers to clean up after their Plugins after an uninstall. I understand that sometimes Users want to keep their data but delete the Plugin to reinstall it, so developers don’t delete files/databases but, at the same time, there’s no option to delete everything when Users DO want EVERYTHING deleted. Leaving unnecessary files and such.

          A simple API for developers to hook into to PROVE they’re Plugin delete’s everything upon Delete – plus a “go back to previous version” option could solve the problem for both parties.

    • TheHandOfCod 12:25 pm on May 28, 2014 Permalink | Log in to Reply

      I think the main thing that would help is to improve the search. If you do a search on ‘Form’ for example the first plugin shown has a lower star rating then plugins displayed later in the list. As mentioned above the Tag Cloud leaves a lot to be desired also.

      Another idea could be to allow installed plugins to be associated with custom categories on the installed plugins page? And allow bulk activate/deactivate by category? Paging would be good as well, with the ability to change the number plugins seen per page. This would help with not losing the menu when scrolling through more than a few plugins.

      I think the ecommerce style approach could be confusing as it might look to ‘new’ users as though they were buying plugins rather than installing free plugins from the repository. However I do like the way that Pippin Williamson displays extensions for EDD https://easydigitaldownloads.com/extensions/, and maybe taking direction from this style could be a good idea.

    • earnjam 1:19 pm on May 28, 2014 Permalink | Log in to Reply

      Something else I’d like to see related to the plugins page is some discussion on #14569

      In multisite it’s pretty confusing having themes and plugins operate in different ways (activate vs network activate vs enable vs network enable). Any patch that deals with that would involve modifications to the plugins page. But well before that, I think there should be discussion about the merits of it and how it would be handled in the first place.

      • Ipstenu (Mika Epstein) 1:23 pm on May 28, 2014 Permalink | Log in to Reply

        That said, plugins and themes are vastly different on multisite in how they behave. If you network activate a theme, it’s available on all sites for possible use. If you network activate a plugin, it’s ON for all sites. But I would suggest this is out if scope for a plugin page cleanup.

        • earnjam 4:23 pm on May 28, 2014 Permalink | Log in to Reply

          That’s my entire point. WHY should they even be acting differently? If you can make themes available to only certain sites, why not make plugins function the same way?

          I agree that it’s beyond the scope of what was in the OP above (no mention of multisite), but as long as we’re talking about changes to the plugins interface, and we really want to pay more attention to multisite on this release cycle (as has been stated a few times), I think that this would be a valid discussion to have. Maybe not here, maybe IRC or trac, but definitely somewhere.

          • Ipstenu (Mika Epstein) 4:33 pm on May 28, 2014 Permalink | Log in to Reply

            WHY should they even be acting differently?

            Because… a theme is not a plugin. But the issue is language, not behavior here :) We have a trac ticket on this I thought, but can’t find it.

            You can only have ONE theme active on a site at a time, right? So a ‘Network Actived’ theme is actually a ‘Network available’ theme.

            On the other hand, you have 50 plugins on a network, and 10 should be permanently on for all sites (network active) and the other 40 should be available.

            So I feel it’s outside the scope of enhancing the plugins pages, because I feel the issue lies not in the activation and handling of plugins, but in the terminology used :)

            • earnjam 7:34 pm on May 28, 2014 Permalink

              Oh, I definitely agree that the language could use some improvement. I’ve seen that ticket somewhere too. Maybe the discussion on #18301 ?

              But I think seeing this only as a language problem is a narrow way of viewing it. Just because you have 50 plugins installed on a network doesn’t mean you want all 50 to be available to all of the sites. Just as if you have 50 themes installed, doesn’t mean you want all 50 themes available to all sites. With themes, you can enable their availability for activation on an individual or network-wide basis. With plugins, once they are installed, they are all available for activation all the time.

              Again, I agree it is outside the scope of the original post here, but I don’t agree that it is outside the scope of general enhancements to the plugins page because this pertains to what plugins are available for activation…which is exactly what shows on the plugins page. :)

            • Ipstenu (Mika Epstein) 8:30 pm on May 28, 2014 Permalink

              Ahh so you’re thinking the granual control.

              https://wordpress.org/plugins/plugin-commander/ type stuff?

              YES, that should be there. I thought you meant something else!

            • earnjam 8:39 pm on May 28, 2014 Permalink

              Yes! That’s what I’m talking about.

              https://wordpress.org/plugins/multisite-plugin-manager is a good example, though not a very nice UI.

    • Ipstenu (Mika Epstein) 1:24 pm on May 28, 2014 Permalink | Log in to Reply

      Have you seen the layout for Jetpack modules? That is nice and neat and would be kind of awesome. Imagine if a plugin could use the menu icon for the plugin page like that?

    • Paal Joachim Romdahl 1:45 pm on May 28, 2014 Permalink | Log in to Reply

      Sometimes I install plugins, activate them but they remain unused. When I want to clean up the plugins I wonder which plugins are not used and are alright to delete. It would be nice if there was a signal of some kind showing where and how a plugin is used.

      Also when a plugin requires other plugins or have add-ons it would be nice to add a drop down or something showing the connection of these “child plugins” in connection with the “parent” plugin.

      • michalzuber 4:20 am on May 29, 2014 Permalink | Log in to Reply

        +1

        • The Portland Company 7:07 pm on May 29, 2014 Permalink | Log in to Reply

          +1

          Though there is some serious consdieration that would need to go into how to identify that sort of thing. Maybe even an API required for developers to opt-into. There is such a variety of Plugins – many of which aren’t directly interacted with – that I can’t imagine a way we could measure their used-ness.

          But I agree!

    • Charleston Software Associates 1:49 pm on May 28, 2014 Permalink | Log in to Reply

      I think the API needs to be improved along with this effort. The info server should allow for results to be filtered and sorted. Sort by download count, last updated date. Filter by compatible with my version, etc. My initial investigation was that any filtering needs to first retrieve a search query THEN filter those results which is less than optimal.

      Helping users, whether newbs or WP gurus, find the right plugins would be a big step in easing “plugin frustration”. Anything that stops the typical plugin search pattern of “search, install, not what I needed/broken/insecure, uninstall, repeat” would be a BIG help for myself and many of my clients. Filtering and sorting searches from within the WP Plugin Add New page would be a step in the right direction.

    • hereswhatidid 2:05 pm on May 28, 2014 Permalink | Log in to Reply

      I’d like to see some classification of the plugins in the admin. Something like Front End, SEO, etc… Similar to the Tags list on .org but more generic. This is something that I’ve seen in a few other CMSs and it really helps with the UX when there are a lot of plugins/modules installed.

    • crzyhrse 2:09 pm on May 28, 2014 Permalink | Log in to Reply

      I’d like to see links on the WordPress Plugins page that go to each plugin’s WordPress plugin page. Most often as it is they go to the author’s web page(s).

      To make it easier to look for support, to rate, to donate, whatever…

    • Paal Joachim Romdahl 4:50 pm on May 28, 2014 Permalink | Log in to Reply

      (Originally posted by Spencer Hill http://make.wordpress.org/core/2014/05/06/summary-of-last-weeks-dev-chat-on-4/#comment-14298 )

      I’d (Spencer Hill) like to contribute to Plugin Installer enhancements with @nacin.

      More specifically I believe we need to rethink the visual architecture of that page. Here are a few examples of what I mean:

      Some Plugins *revise* Core features.
      Others *replace* Core features.
      Then some *extend* Core features.
      Others introduce entirely new features that will not, and should not, become apart of the Core.
      And, lastly, some *extend* or *revise* *Plugins* themselves. These are commonly referred to as Add Ons or Extensions (like WooCommerce Extensions).
      As a User, viewing the Plugins screen these all blur together and there’s no way to filter between them. I find myself accidentally installing Plugins with duplicate features. And there is no way to see the relationship between some Plugins like “Add On” Plugins (which are the ones that extend or revise Plugins themselves).

      I’ve prepared a mockup that can be viewed here: https://drive.google.com/a/theportlandcompany.com/folderview?id=0B8-MGuUsAa39dHdRa1I0SG9yZVE&usp=drive_web

    • Jon Brown 6:04 pm on May 28, 2014 Permalink | Log in to Reply

      Almost completely absent from this discussion seems to be “Getting better user feedback published back to the repo”.

      In order to “Help users to discover the plugins they need” we need better data published to .org.

      I’ve been pushing this for a while so realize there are auth issues in publishing reviews/compatibility reports directly from a .org install back to wordpress.org. However, I really think part of this push should be figuring out how to encourage plugin feedback or mining what data we can better. I do like the idea of reporting “Active Installs” for popularity instead of downloads, but we need to focus on this a bit and figure out what we can do.

      • Michael Beil 8:44 pm on May 28, 2014 Permalink | Log in to Reply

        I agree Jon.

      • The Portland Company 7:12 pm on May 29, 2014 Permalink | Log in to Reply

        +1 – @nacin mentioned he was responsible for this in an IRC a couple weeks ago.

        I don’t know how this could be done while respecting privacy, but I know that when I’m searching for a new Plugin I install, uninstall. Find a new one; install, uninstall. Find another; install, uninstall. Until I narrow it down to one or more. So if there was a way we could share that data with other users so they don’t waste time on crap-Plugins I think that would be valuable. But that may be beyond the scope of what we’re trying to do here and that’s understandable!

      • Andy McIlwain 9:20 pm on June 3, 2014 Permalink | Log in to Reply

        +1. I’m really interested in seeing some quantitative+qualitative feedback. Particularly if we can identify some standout pain points.

    • Arnan de Gans 7:10 pm on May 28, 2014 Permalink | Log in to Reply

      While I think there are some good points here, at the same time I’m also thinking the current listing of installed plugins doesn’t need fixing as it works perfectly fine as it is.

      • The Portland Company 7:16 pm on May 29, 2014 Permalink | Log in to Reply

        I have to disagree. I mean, it’s *functional* but has a few *major* issues relating to security and database optimization:

        • Many Plugins don’t clean up after themselves properly during Delete.
        • Users should know, on Delete, whether or not a Plugin is going to remove absolutely everything it created (database or files) or leave something behind.
        • - If it does Users need to be given the option to say “No, I want you to remove everything. Don’t force me to let you leave crap behind in my file structure and DB”.
        • Ipstenu (Mika Epstein) 3:11 pm on May 30, 2014 Permalink | Log in to Reply

          That is not all that easy. The way plugins clean up is via an uninstall script, and at this time, without a total overhaul of not just the plugin page but plugin install processes, it’s not feasable. Should we look to it long term? Maybe… It’s a messy idea, and that’s much of why we left it in the hands on the plugin devs.

          Sadly that’s also why it’s not as common as it could be.

          1) Plugins ALWAYS delete the plugin folder files on uninstall
          2) Plugins are ENCOURAGED to delete tables/options on uninstall
          3) Plugins RARELY delete the ‘other’ files it adds (like advanced-cache.php, extra uploads etc)

          Deleting the non-plugin-folder files is super messy and not something I’d want to see for all plugins. Like a gallery? Imagine if NGG nuked all your images on uninstall. You’d be livid :) That’s always going to be a manual call there for sanity.

          Deleting the tables? Even then, it has to be thoughtfully done per plugin. Take those role editor plugins. You would, theoretically, want them not to delete but to reset on uninstall.

    • Dovy Paukstys 8:14 pm on May 28, 2014 Permalink | Log in to Reply

      I’d like to see a safety feature built in. When you update a plugin, move it to a store directory. A cron task is setup to delete it in say 8 hours. If the user is not happy with the update or it breaks something, you can restore to previous. That would help the user side of things.

      • Greenweb 9:19 pm on May 28, 2014 Permalink | Log in to Reply

        That would assume that any data and or data schema related to the old version was not changed on activation of the new plugin.

        • Jon Brown 1:24 am on May 29, 2014 Permalink | Log in to Reply

          +1 – reversion is complicated.

          If a user needed/wanted to they can download an older version from .org and manually install.

        • Ipstenu (Mika Epstein) 3:14 pm on May 30, 2014 Permalink | Log in to Reply

          That’s actually less complicated. Since DB/data changes usually happen with an intentional click after upgrade it could be safe ‘enough.’

          There are plugins that make major db structure overhauls, and yes, that would be problematic. Still, a file dump to flip to the last version requires one thing we don’t have :) What was the OLD version?

          Copying the folder to a plugins-old or plugins-backup folder might do it, though we’d have to come up with a way to delete THAT after x time (NOT 8 hours, more like 5 days). Feasible, though. I’d love to see a plugin do that so we could experiment with it!

    • David Lingren 10:49 pm on May 28, 2014 Permalink | Log in to Reply

      I hope there will be a corresponding effort to improve the Plugin Repository as well. For example, a plugin-specific Support Forum “Search” facility that would only look at topics within the current plugin. The Repository-wide search is useless in many cases.

    • michalzuber 4:18 am on May 29, 2014 Permalink | Log in to Reply

      Would be cool to have Recommend (for example https://i.imgur.com/bLyfW4X.png from https://play.google.com/store/apps) showing what my friends favorited on WP.org

    • Dan Griffiths 8:10 pm on May 29, 2014 Permalink | Log in to Reply

      Agree with A LOT of the proposed changes… probably the single most useful I can think of would be the addition of native plugin dependency support. That said… we’ve already lost the install_themes_tabs filter (which I used)… please don’t lose the install_plugins_tabs filter too… or at least provide suitable replacements. It might not be common use, but there are valid reasons for needing to extend/modify the theme/plugin tab bars…

    • Josh Pollock 12:18 am on May 30, 2014 Permalink | Log in to Reply

      I had a user today report that my plugin couldn’t be installed via the theme installer since it didn’t have a valid style.css. This may seem silly to experienced users but how clear is the difference between plugins and themes to new users?

      It seems to me that as long as the plugin installer, should, if the uploaded file fails plugin validation, check if it passes theme validation and if so point the user to the right installer and the theme installer should do the same thing as well. This would take one pain point away from learning our platform and turn a frustration into a learning experience.

      • Ipstenu (Mika Epstein) 3:17 pm on May 30, 2014 Permalink | Log in to Reply

        The error message there should be a little better. “The zip you have attempted to upload does not appear to be a THEME.” and then possibly a check to see if it has plugin headers so we can correct people?

    • sffandom 10:29 pm on May 30, 2014 Permalink | Log in to Reply

      ” Incorporate things like ratings, support stats, age, usage stats, and update frequency in the relevancy scores.”

      That would be a very bad idea. If a new plugin is being frequently updated to compete with an older, more robust plugin, then the user would be misled into thinking the new plugin has an advantage.

      The same goes with ratings. How do you compare a 4-star rating based on 5 feedbacks with a 4-star rating based on 500?

      And given how some developers mark support threads as “Resolved” when they are NOT resolved, the support stats would be useless.

      You touched on this problem here: “Improve the algorithm used for averaging ratings, to smooth out errors for plugins with only a handful of ratings.”

      Yes, but ratings in general are an unreliable metric for quality, suitability, or matching user needs.

      Compatibility is really not very helpful in many cases. A lot of old plugins work just fine with the current version of WP. What would be helpful is a catalog of reported errors. If a plugin is really incompatible with a new version of WP there should be a way for people to report that and for the plugin dashboard to say, “400 users reported compatibility issues with this plugin from version X on up.” Not perfect, but much better than the current system.

      • Alex Shiels 5:48 pm on June 7, 2014 Permalink | Log in to Reply

        I agree that ratings are unreliable metrics. And that update frequency, download counts and active installs could lead to an undesirable feedback loop that unfairly promotes a limited subset of plugins.

        I’m not suggesting that any of those things be used as the sole metric for ranking search results. Merely that they be incorporated into the ranking algorithm provided that the end result does in fact improve search quality. We do have access to engines and expertise in that area.

    • Pete 11:09 am on July 25, 2014 Permalink | Log in to Reply

      The ability to categorise all my favorited plugins would be awesome. As it is now I have 10 pages of my favorited plugins with no ability to search/browse/categorise them in any meaningful way.

  • Mike Schroder 9:32 am on April 15, 2014 Permalink | Log in to leave a Comment
    Tags:   

    Last Week in WordPress Core 

    Hello there! This is Last Week in WordPress Core for the week of April 8-April 14. Similar to last week, commits are included up to RC2, which was released today. In addition, maintenance releases 3.8.3 and 3.7.3 are available, and automatic updates are rolling out.

    Customizer

    • Widgets: Properly handle widget settings when activating a previewed theme. [28124] #27767
    • Widgets: Account for a sidebar with no container to which classes can be added. [28100] #27780
    • Custom Headers: Fix image ordering. [28102] #27791
    • Custom Headers: Fix cropping when working with large images. [28101] #27790
    • Add color scheme support for widget choosers. [28122] #27793

    Theme Installer

    • Improve route handling and make ?theme= work. [28123] #27708
    • Revert to proxying through PHP for WordPress.org API requests to ensure we have valid installation nonces. [28126] #27798

    TinyMCE

    • Update TinyMCE to 4.0.21.1. [28066] #27744
    • Update TinyMCE paste plugin to the latest development version. Improves Pasting from Word to remove inline comments and change tracking. [28089] #27771
    • Ensure vertical resizing and menubar show/hide are set to default for each TinyMCE instance. [28059] #27724
    • Stabilize MediaElement within TinyMCE, and avoid adding undo steps when the body of a wpView changes. [28084] #27389
    • Improve fallback compatibility for wpViews with IE7 and 8. [28062] [28063] #27546

    Media

    • In the Image Details modal, remember the last state of the advanced toggle. [28125] #27366
    • Add hooks for wpeditimage TinyMCE plugin and Image Details modal. Includes wp.media.events, which is intended to be a global media event bus. [28095] #27698
    • Apply new add_image_size() cropping preferences to all sizes when image is saved in editor. [28072] #19393
    • Fix tabbing out of the title field on Media->Edit Media screen. [28069] [28070] #27750

    Internals

    Externals

    For the complete list of commits to trunk, check out the log on Trac. Release is scheduled for this Wednesday, so, the best way to help is to test! Please let us know if you run into problems in the Alpha/Beta forums or on trac.

    Thanks to @azaozz, @dd32, @DrewAPicture, @ehg, @GaryJ, @gcorne, @helen, @jesin, @johnbillion, @kerikae, @kpdesign, @mattheu, @matveb, @melchoyce, @morganestes, @nacin, @ocean90, @Otto42, @pavelevap, @redsweater, @ryelle, @scottlee, @SergeyBiryukov, @siobhan, @westonruter, @wonderboymusic, and @yoavf for their help this week!

     
    • Stagger Lee 5:03 pm on April 15, 2014 Permalink | Log in to Reply

      I am using WP Beta tester plugin, latest nightly and video playlist is still missing. Audio playlist is there. Is it normal for this beta moment ?

      • Mike Schroder 6:44 pm on April 15, 2014 Permalink | Log in to Reply

        Have you uploaded any videos? The “Create Video Playlist” option should appear on the left in the “Add Media” modal after you have videos uploaded to compile into a playlist.

    • Stagger Lee 8:23 pm on April 15, 2014 Permalink | Log in to Reply

      Yes i see now, thank you. Tried to upload it with FTP client first. (C/P, localhost) Still a bit confusing for beginners.

      Do you plan to make same styled playlists for Youtube videos ? I know there are plugins for it, but it is core code, style is fine and URL option is already there.

  • Mike Schroder 8:55 pm on March 19, 2014 Permalink
    Tags: ,   

    Last Week in WordPress Core 

    Hey Everyone! This is Last Week in WordPress Core for the week of March 10–16. It’s been a busy week after Beta 1. Here are some highlights from the week:

    • Appearance: Bring the theme browsing experience from 3.8 to the theme installer. [27499] #27055
    • Customizer: Add header image uploads with cropping to the customizer. [27497] #21785
    • Plugin Management: Restyle the plugin install details modal to match the rest of the admin. [27559] #26952
    • Edit Post: Correct the “View Post” button link when changing a post slug. [27508] #16477
    • Admin Colors: Revert [27203], fix color scheme stylesheets. Restores [27111]. [27515] #27175; see #20729.
    • Editor: figcaption should not be treated as a block-level element by wpautop(). [27527] #25646
    • TinyMCE: add internal command and shortcut (Alt+Shift+X) for toggling <code>. Define a button that can be added to any toolbar as wp_code. [27545] #6331
    • Permalink Settings: Don’t show “update your .htaccess now” if nothing needs to change. [27549] #19268.
    • Query: In WP_Query::get_queried_object(), account for pre_get_posts by checking for tag when tag_id isn’t present. Tags still need to be rolled up into tax_query.  [27511] #27362
    • Filesystem: Update request_filesystem_credentials() to handle the correct ssh value of FS_METHOD. [27546] #27265

    Widget Customizer:

    • Move widget area sections to bottom, as a theme can have a lot of widget areas and we don’t want to bury other sections. [27541] #27401
    • Introduce a customizer processing state to prevent saves while updates are occurring. [27540] #27390
    • Make temp hooks permanent. New hooks are: dynamic_sidebar_before, dynamic_sidebar_after, dynamic_sidebar_has_widgets and is_active_sidebar. [27543] #25368

    Media:

    • Start embedding functional audio/video players in the editor, instead of placeholders. [27528] was reverted in [27530] but added back this week in [27615]. Whitelist media types by browser. [27539] [27542]. See also [27534] [27535] [27536] [27537] [27538] and others. Everything is contained in #27389.
    • The Image Editor should apply changes to custom image sizes by checking registered image sizes. [27522] #19889
    • Remove Qik from the oEmbed provider list as it’s shutting down. [27526] #27302
    • Smooth out some display and race condition issues with the media modal loading spinner. [27516] #24859

    XML-RPC:

    • Avoid saving slashed data in XML-RPC’s wp.setOptions. [27551] #22936
    • Allow query strings for servers in IXR_Client and WP_HTTP_IXR_Client. [27552] #26947
    • Include sticky in the struct returned from metaWeblog.getRecentPosts. Using wp.getPosts is preferred and non-WP XML-RPC APIs are no longer actively maintained. This is simply for parity with existing MW methods. [27553] #26679
    • In wp.editPost, Remove all terms in a taxonomy when an empty array is explicitly passed. [27554] #26686

    For the complete list of commits to trunk, check out the log on Trac.

    Interested in joining in? Write or test a patch for 3.9. The goals for this week — besides releasing Beta 2 — are two-fold:

    Thanks to @aubreypwd, @avryl, @azaozz, @bravokeyl, @cfinke, @danielbachhuber, @DrewAPicture, @ehg, @enej, @ericmann, @gcorne, @helen, @jayjdk, @jnielsendotnet, @johnpbloch, @joostdevalk, @jstraitiff, @JustinSainton, @kadamwhite, @klihelp, @kovshenin, @ldebrouwer, @mattonomics, @matveb, @mauryaratan, @maxcutler, @mcsf, @MikeHansenMe, @nacin, @nendeb55, @ocean90, @oso96_2000, @Otto42, @paulwilde, @pento, @rodrigosprimo, @SergeyBiryukov, @soulseekah, @tlovett1, @westonruter, @wonderboymusic, and @wpsmith for their help this week!

     
  • Mike Schroder 10:29 am on March 13, 2014 Permalink
    Tags: ,   

    Last Week in WordPress Core 

    Hi there! Welcome to Last Week in WordPress Core for the week of March 3–9. By now, you’ve heard that WordPress 3.9 Beta 1 is available! Thank you for your hard work this last week. Now we’re done adding new enhancements, and on to bugs. Your help is appreciated as we continue to test and squash bugs on the way to a stable RC.

    There are a couple important things that landed on Monday that are not covered in this post, but shipped in beta. Namely, please test the Theme Install screen refresh and the ability to crop headers from within the Customizer.

    Admin:

    • Widgets: Add widget management to the customizer. This brings in the Widget Customizer plugin. [27419] #27112
    • Admin Menu: Introduce a .dashicons-before CSS class and use it in the admin menu. Lets you use a Dashicon before an element without copying the entire .dashicons styling to your :before styling. [27418] [27425] [27444] [27482] #26630
    • Editor: Show “View Post” for any post the author can read. This expands it to private posts and matches the logic in the toolbar. [27483] #27059

    Media:

    • First pass at bringing the Image Editor into the media modal. Please test me! [27445] #21811
    • First pass adding a loading indicator to the Media Library. [27438] #24859
    • Allow $crop in add_image_size() and set_post_thumbnail_size() to receive crop anchors (top, left, right, bottom, center). [27472] #19393.
    • Add subtitle support to Video editing in the Media Modal. [27481] #27016
    • Do not output default gallery styles if the theme has opted into HTML5 galleries. [27396] #27045; see #26697
    • Add a class attribute to the caption shortcode to allow additional classes to be specified. [27404] #25295
    • Add playlist_styles and wp_playlist_scripts filters to allow users to roll their own playlist themes. [27486] #26631 & [27488] #26631

    TinyMCE:

    • Update TinyMCE to 4.0.18. [27387] #24067
    • Add TinyMCE placeholders for audio and video shortcodes and provide a UI to both edit shortcode attributes and replace the src media file in an audio or video shortcode. Also, a flurry of improvements and fixes to them, visible in the full changelog. [27411] #27016
    • Add a Ctrl+K shortcut to open the linking dialog, which is the “de-facto standard”. [27449] #27305
    • Add the <hr> plugin and button to the toolbar. [27428] #27159
    • With drag-and-drop uploading, support multiple editor instances, limit to IE10+, and other small fixes. [27378] [27372] [27464] #19845
    • When parsing a caption shortcode, recreate missing width attributes using the image tag’s width. [27426] #23103
    • Restore the “link” button state to disabled by default and enabled when text or image is selected. Remove the (recently added) default link plugin; not needed. [27447] #27309

    Templates:

    • Add has-post-thumbnail as a post class. [27429] #18804
    • Rename the new page_templates filter to theme_page_templates, and pass it a post object for proper context. [27470] [27471] #13265
    • Introduce get_the_permalink() as an alias for get_permalink(). This better aligns it with other the_* and get_the_* function pairs. [27409] #24164
    • Let get_the_date() accept a post object. [27380] #13771
    • Add the ability to short-circuit wp_nav_menu() via the pre_wp_nav_menu hook. [27386] #23627
    • Better plural handling for labels in wp_generate_tag_cloud() / wp_tag_cloud(). [27376] #27262, see #7989, #14424

    Multisite:

    • Incremental improvements and bug fixes with the multisite load process. Please test your networks! [27406] [27439] [27407] #27003
    • Fix bulk activation of network-only plugins. [27413] #26487

    Query:

    • Add has_password and post_password query variables to WP_Query. has_password true means posts with passwords, false means posts without. post_password can query for posts with a particular password. [27395] #20308
    • Allow a posts_per_rss query variable to be set to override the posts_per_rss option. [27456] [27455] #25380
    • Allow get_page_by_path() and get_page_by_title() to accept an array of post types. [27423] #24763

    Internals:

    • Allow for custom authentication handlers for all requests. Turn the logic used by wp_get_current_user() into a determine_current_user filter. [27484] #26706
    • Allow the role attribute in kses for all elements. [27388] #24098
    • Add a pre_set_theme_mod_$name filter to set_theme_mod(), modeled after pre_update_option_$option in update_option(). [27393] [27402] #14721.
    • Improve HHVM compatibility by eliminating some of our last remaining create_function() calls and making OBJECT a case sensitive constant. [27373] [27374] [27465] #14424 [27377] #27231
    • Pass $reassign parameter to delete_user and deleted_user actions. [27462] [27466] #23057
    • Bail early from shortcode functions if no delimiter is present. It’s the little things; performance results on-ticket. [27394] #23855
    • Update PHPMailer to 5.2.7 from 5.2.4. Includes two trivial modifications for WordPress (no impact to plugin developers); see the commit message. [27385] #25560
    • Use SSL when linking to WordPress.org. [27469] #27115

    For the complete list of commits to trunk, check out the log on Trac. Interested in joining in? Write or test a patch for 3.9.

    Thanks to @adamsilverstein, @akeda, @avryl, @bassgang, @bigdawggi, @bobbravo2, @bpetty, @bradt, @celloexpressions, @coffee2code, @danielbachhuber, @dd32, @DJPaul, @DrewAPicture, @empireoflight, @ericlewis, @ericmann, @frank-klein, @gcorne, @genkisan, @gradyetc, @hakre, @Hanni, @Jayjdk, @jenmylo, @johnregan3, @jorbin, @JoshuaAbenazer, @kadamwhite, @kasparsd, @Kopepasah, @kovshenin, @kpdesign, @lpointet, @markjaquith, @mcadwell, @melchoyce, @michael-arestad, @mikecorkum, @mordauk, @nacin, @obenland, @Otto42, @pavelevap, @Rarst, @rhyswynne, @ricardocorreia, @rmccue, @robmiller, @seanchayes, @SergeyBiryukov, @shaunandrews, @simonwheatley, @sirzooro, @tanner-m, @TobiasBg, @tomauger, @topher1kenobe, @topquarky, @toszcze, @westonruter, @wokamoto, @wonderboymusic, @zbtirrell, and @zodiac1978 for their efforts this week!

     
  • Frederick Ding 12:03 am on July 23, 2013 Permalink
    Tags: , ,   

    Migration update: cron importer 

    Following last week’s update about the WP_Importer_Cron approach to writing importers and running import jobs, I’ve been steadily transitioning code from the current non-stateful, single-execution plugin to a stateful, step-wise process (#327).

    At the same time, I needed to separate presentation from logic/backend processing (#331) — something that @otto42 also recommended — in two ways:

    • Removing direct printf(), echo statements that were used by the WXR importer (example)
      and changing them to WP_Error objects (example of fatal error; of non-fatal warning)
    • Handling uploads and UI choices in a separate class

    Why must this be done now? Well, asynchronous tasks differ from PHP scripts directly responding to a browser request — we can’t depend on having access to submitted $_POST data, nor can we directly pipe output to the user. This change would also make it easier to understand what the code is doing from reading it, and to test programmatically.

    One dilemma I’ve encountered: how best to store the parsed import XML file. Since each step of the import (users, categories, plugins, etc) runs separately, we must…

    1. store all of the parsed data in variables, which are serialized into an option between runs
      (obviously, a huge amount of data for which this may not be the most robust or efficient method);
    2. re-parse the XML on each run
      (currently, parsers handle all parts of the XML at once, which means unnecessarily duplicated effort and time);
    3. modify the parsers to parse only part of the XML at a time; or
    4. split the XML file into chunks based on their contents (authors, categories, etc) and then feed only partial chunks to the parser at a time.

    Any thoughts? Solving this problem could also help the plugin deal with large XML files that we used to need to break up by hand before importing. (The Tumblr importer doesn’t have the same problem because there is no massive amount of data being uploaded at the beginning.)

    I haven’t yet finished transitioning all the steps; I’m afraid it won’t be possible to use this just yet. Before next Monday, I should have a downloadable plugin that’s safe to try.

     
    • dllh 1:34 am on July 23, 2013 Permalink | Log in to Reply

      You’ll want to be careful about trying to store data in options. For example, in an environment that’s using memcached, which has a size limit, you can blow up sites by trying to store too much data in options, so it’s not necessarily a matter only of efficiency.

      Also, if you use options, I imagine you also have to use something like an extra tracking option to know which parts of the import you’ve handled. This is just begging for race conditions.

      I wonder if there’s anything you can do using FormData to split the file into chunks client-side and assemble them server-side. I’m not sure what browser support is like, and I worry it’d be pretty brittle. Just thinking aloud.

      • Frederick Ding 1:44 am on July 23, 2013 Permalink | Log in to Reply

        You’re completely right about the caching implications — I’ve all but eliminated possibility #1.

        I was thinking about the possibilities of client-side file splitting, too, even though it’s not the most robust or reliable way. Last week, I looked briefly at https://github.com/blueimp/jQuery-File-Upload which supports chunking/multipart — but that’s not quite what we’d need, is it? The browser would need to operate along XML node boundaries, not just file size. (I’d be intrigued if it’s possible to utilize a browser’s native DOM engine to do that…)

    • Ryan McCue 1:50 am on July 23, 2013 Permalink | Log in to Reply

      So, with regards to XML parsing options: splitting the XML file is something that absolutely should not be done. You can do it, but only if you use proper XML serialization/deserialization, which is likely going to take up a chunk of time.

      Reparsing is a bit of a pain too, since the XML parsing usually takes up the largest amount of time there. The best bet with regards to memory is to use a SAX-style parser which streams the XML, whereas a DOM-style parser will read it all at once. SimpleXML is a DOM-style parser, so you should avoid that for performance, whereas xml_parse is SAX-based.

      I’m biased as the lead developer of it, but you could use SimplePie, which a) is built into core, b) has built in caching (via the transients API, but you’d probably want file-based caching here) and c) natively parses RSS (given that’s its job), which WXR is based on. This handles picking a parser all internally, although doesn’t use a stream parser due to internal implementation details, so you may want to stick away from it for that. I’m relatively certain I could write a parser (in `WXR_Parser_*` form) in a few hours, so it should be easy to do.

      At the least, I’d take a look into how SimplePie handles caching. Basically, it’s a giant serialized array, usually saved to a file (although in WP, this uses transients).

      (If you do want to take the full SimplePie route, I’d love to help, most likely in a consulting role. It’d simplify the `WXR_Parser` classes significantly by moving the XML parsing out of those.)

      I’ve spent a fair bit of time messing with XML parsers, so feel free to pick my brain on this! :)

      • Ryan McCue 1:55 am on July 23, 2013 Permalink | Log in to Reply

        (What you really want here is a resumable SAX/pull parser, with some way to persist state across processes. With `xml_parse`, you should be able to feed in data as you stream it in, but you’ll still need to keep the data in memory since I don’t know of any streamable serialization in PHP. Please note that if you do use your own code here, there are security considerations that will need to be discussed. Your mentors should be able to help with that.)

      • Frederick Ding 2:51 am on July 23, 2013 Permalink | Log in to Reply

        I should have remembered that you’re an expert on XML!

        I’m not familiar with the differences between SAX/pull/DOM, but if I’m reading this right, then pull — with a way to persist state — would be the most appropriate model to follow. (In PHP, XMLReader appears to be a pull parser that’s enabled by default since PHP 5.1.2 — so I’d just need to write a WXR_Parser_* to take advantage of it.)

        Thanks for pointing me in this direction!

        Edit: Actually, scratch my eagerness to use XMLReader. It appears to be weakly documented and most of what I can see involves using SimpleXML/DOM on nodes…

        • Ryan McCue 6:43 am on July 23, 2013 Permalink | Log in to Reply

          XMLReader or SAX are the ones I’d go for. There’s an IBM DeveloperWorks article that might help with that.

          The way I’d handle it is to keep state of where you currently are in the stack. WXR_Parser_XML uses this concept to handle it, but uses the SAX parser instead. Conceptually, the way it keeps track of the position is the way you’d want to do it (although there’s a bunch of cases it doesn’t handle).

          WXR_Parser_XML isn’t the best though, since all this data is loaded into memory and then pushed to the database later. Although it means you have cross dependencies, I’d consider inserting posts/etc as you go, rather than bunching them all up. This is a pretty fundamental rework of the internal parsing API, but it’s one that you’ll need for this sort of thing.

          Personally, I’d create two objects (a parser and a data handler) and use dependency injection to ensure that you still have a weak coupling between the two.

          (As a note, SimplePie also uses the SAX parser, but loads all the data into memory because it has to. This is a case where it’s a much better idea to use XMLReader directly.)

          Regarding XMLReader documentation, are there any specific examples? I’m happy to assist with specifics here, ping me at my email (see footer) if you’d like.

        • Ryan McCue 6:51 am on July 23, 2013 Permalink | Log in to Reply

          Also, the reason people are using XMLReader with SimpleXML/DOM is because they prefer the SimpleXML/DOM API, and the performance using the hybrid is better than straight SimpleXML. Personally, I’d stick with the one rather than switching, because there are some performance concerns with that.

  • Frederick Ding 12:00 am on July 16, 2013 Permalink
    Tags: ,   

    Migration update: WXR importer 

    This week, I began work on the next phase of my project: fixing up the WXR importer plugin. A number of developers, including Jon Cave, Peter Westwood, and Andrew Nacin have been maintaining this plugin since at least May 2010.

    I forked the code from the plugins repository into my GSoC Subversion directory in preparation. It’s taken a while to test this (manually) against XML files from existing sites, so that I can see under what circumstances it fails to complete the import or perform to expectations, and what can be done. Trac tickets and forum posts have been informative as well. (See the linked posts for the results and observations.)

    I’ve also run the unit tests that apply to the importer plugin; however, the test cases are generally small (indeed, the biggest XML test case is 26 KB, titled small-export.xml) and don’t trigger the kinds of issues that importing dozens or hundreds of posts and attachments—with WXR files of megabytes in size—does.

    So, the first task at hand is breaking up the process—which currently executes in one step with little indication of progress—into discrete chunks that can run in separate, stateful (stepwise) requests.

    A chat with my mentors has pointed me in the direction of WP_Importer_Cron, which was first developed for other importers that need to make external API calls (e.g. Tumblr Importer) potentially subject to rate constraints. There are some parallels between “external API calls” and “remote attachment fetching”, which is why this can be a suitable approach for fixing the timeout issues that present with the current WordPress importer. After the process is discretized, showing progress (an enhancement long overdue) will be easier.

     
    • dllh 1:07 pm on July 16, 2013 Permalink | Log in to Reply

      I’ve worked quite a lot with imports and am glad to see you doing some work on this front. One of the problems I think you’ll wind up running into with attachment handling is backfilling. You have to maintain a list of attachment ids and their parents, and the ids can change as the import process goes on and would-be id collisions are averted. Further, there are sometimes images that are referenced in posts but not attached to them. So a backfill process that takes these things into account is important.

      So farming fetching itself out to cron isn’t going to solve the problem, though it may help with timeouts. Some issues I foresee:

      • On sites with little traffic, the images will basically never import, or people will perpetually file bug reports that it’s not finishing, when in fact, they just need a few hundred visits to cycle through fetching all their images.
      • There’s no built-in way to track status of a given image. If you try to use something like options to track status, you’ll wind up with too-big options (potentially bad for sites using memcached) or find that race conditions prevent proper saving of the data.
      • What happens if images start to be processed before the import itself is done? This becomes an issue when an attachment that appears near the top of the export is fetched before a post near the bottom of the export that references it is fetched. A backfill process that’s timed with the image fetch won’t yet have that late post to check for a reference to fix up.
      • What happens if the import is halted partway through? This wreaks havoc with backfilling in particular.

      It occurs to me (I’m shooting from the hip here, so it’s not terribly well thought-out and may be stupid) that you could possibly register a custom post type to get around some of these issues. The process would look something like this:

      1. When the import starts, a unique taxonomy term in a taxonomy registered to the new CPT is created.
      2. Every time the importer encounters an image, it creates a new post of type CPT containing relevant data and with the unique taxonomy term applied. No fetching is done yet. (Or, fetching is done in the background and a post meta is toggled to represent the state of the image progress. So on initial creation, it’s “pending,” when the fetch is in progress, it’s “downloading,” then it becomes “pending_backfill,” then “backfilling,” then “finished” (and “failed” when applicable, which could possibly be useful in diagnosing and fixing issues with the import after the fact — as the CPT data will include post parent, a nice little list of problematic posts could be generated so that the user could fix things up rather than just hoping for the best).
      3. When the importer is finished, it queues a cron job to check for posts of type CPT in the given taxonomy with the unique term.
      4. That job fetches N CPTs at a time and fetches/backfills them (after changing their status in meta so that future jobs don’t scoop them up). If there are any left, it fires another job to get the next batch. Maybe there’s an option stored to prevent doubling up on jobs (or to allow multiple jobs without collision).
      5. When there are no more CPTs with the unique taxonomy term, the import is finished, and the status screen can be updated and an email sent.
      6. If fetch/backfill jobs seem to be languishing, we can report it on the status screen, and perhaps, if they seem to have been stalled for a while, we can have a button you can push to manually fire the job off and kickstart the process.

      I think something along these lines would allow for better display of data about the status of the import as well as less brittle handling of attachments. It won’t necessarily be the fastest process in the world, and the implementation of a CPT for these purposes feels a little weird to me, but otherwise, I’m not sure there’s a great way to track the status of attachments without running into issues with either option sizes or race conditions (trust me — I’ve dealt with this).

      For what it’s worth, I’m not a fan of javascript-driven approaches. JS is fine for polling the status and updating the UI, but as the thing that triggers and tends the process, it seems weak. What if you close the page and the js hasn’t managed to store proper state remotely, for example? Better to have something that runs properly whether you’ve got the import page loaded or not.

      Well, that was a mouthful. Even if my napkin proposal is a crackpot idea, maybe it’ll spark some useful thoughts. Good luck with it. :)

      • Samuel Wood (Otto) 10:48 pm on July 16, 2013 Permalink | Log in to Reply

        WP_Importer_Cron handles a fair amount of the cron work by making the entire import happen as a cron process. Essentially, I first wrote it not because of rate limits, but because a) you run into the time limit on PHP processes when you have a lot of work to do and b) Tumblr is slow as heck in responding to API requests.

        The way you use it is to make your importer class extend WP_Importer_Cron. It gives you some functions to help with this sort of thing.

        • First is save_vars(), which essentially saves your class’s variables to the options table for usage on the later instantiations. Thus, you can save your progress as class variables, then just call save_vars after each step (like after you import a post, for example). This keeps track of your progress. On later runs, your vars are automatically loaded into the class instance for you, putting you back where you were.
        • Next is have_time(), which simply says whether you’re out of time or not. If it returns false, then you’ve exceeded 30 seconds (default) and you need to stop doing things for now.
        • There’s also schedule_import_job( $callback, $args ), which you can call when you’re ready to begin the import. It schedules a job to run every minute, and it will keep running once a minute (or so) until the job is done. How does it know you’re done? Well, you give it a function callback to do the actual job of importing. When that function returns true, the job is considered complete and the cron schedule is removed.

        So what WP_Importer_Cron gives is a way to start a job to run once a minute, a way to check for current run time expiration, and a way to save internal class variables persistently from one run to the next. Plenty enough to build a long-running process on for importing things slowly. You can use a separate importer class from the user-data-gathering pieces, if you like, for simplicity of purpose. Or you can combine the two like the Tumblr Importer does. I’d recommend the first one, the Tumblr Importer is a bit confusing in that respect.

        • Frederick Ding 11:20 pm on July 16, 2013 Permalink | Log in to Reply

          Hey @otto42, thanks for chiming in here!

          Your clarifications for how (and why) WP_Importer_Cron works are really helpful — I wrote some notes for myself earlier today for what I figured by reading its source.

          If you don’t mind, I’ll also write some PHPDoc for the class and its methods building on what I now know, as I start separating pieces and extending WP_Importer_Cron.

          • Samuel Wood (Otto) 6:18 pm on July 17, 2013 Permalink | Log in to Reply

            Don’t look too awful much at how the Tumblr Importer works. It’s not great in that respect. Instead, think about how your process should work and then just use those three functions from the class to run periodically. You’ll probably have better luck that way.

      • Frederick Ding 11:16 pm on July 16, 2013 Permalink | Log in to Reply

        @dllh Wow, thank you for an incredibly thought-out suggestion — there’s a lot to digest here.

        From what I’ve digested of the Tumblr Importer and WP_Importer_Cron source, it stores a counter of the number of posts and other objects imported thus far and the number in total — in an option. I suppose this isn’t as comprehensive as what you’ve proposed with the custom post types — but I’m also not comfortable creating so many transient objects… definitely something to think about.

        As for backfilling: from what I can tell, in WXR files where everything was exported, attachments come first (in another case, attachments and menu items were mixed together but all before posts). I’m not sure whether this is just in the case of the two exports I looked at, or if there is a rationale for this order.

        Certainly the exporter takes great care to make sure that categories and taxonomies are exported with parents before children — I’m not quite certain why attachments would necessarily come before posts, especially in the import process, when posts are necessarily their parents.

        Thanks to WP_Importer_Cron, which suggests a way to hold progress on the server side, the JavaScript would only be for polling the status (and making at least enough requests for cron jobs to be triggered). It wouldn’t be responsible for holding state at all. I need to write a separate response to @otto42, whose comment was just posted as I was writing this.

        Thanks for your input! There’s a lot to be considered.

    • Shea Bunge 11:39 pm on July 16, 2013 Permalink | Log in to Reply

      You should definitely check your importer with the http://wptest.io test data – it’s pretty much the most complete set of test data avaliable.

      • Frederick Ding 3:43 am on July 17, 2013 Permalink | Log in to Reply

        Great resource! Definitely will. I’m assuming that all of the attachment URLs referenced are directly accessible (for attachment fetching)?

  • Andrew Nacin 11:14 pm on November 21, 2012 Permalink
    Tags:   

    If you have a Tumblr blog, can you help test an updated version of the importer? It uses their OAuth API, which requires you to create an application. It’s simple and the plugin walks you through it. Here’s the ZIP file to the beta version. You can report bugs on #22422.

    Edit: The plugin has been released, the beta is over: http://wordpress.org/extend/plugins/tumblr-importer/

     
  • Matt Mullenweg 7:09 pm on August 3, 2012 Permalink
    Tags: meta,   

    You might notice that this P2 has gotten a big head. All of the Make P2s have actually, and like the rug in the Big Lebowski we think it really ties the room together.

    The Get Involved tab has been added, docs have been moved under support, home has been hidden. This isn’t ideal — we’d eventually like to move to more of a verb-oriented navigation system — but it is better than everything under Make being its own island and not really linked to or from the main WP.org side, or to each other. Hopefully it will also let more folks know about how to get involved (I added a link to the Make Core Handbook to the top sidebar widget.)

    I’ll be talking more about some of the improvements to WP.org tomorrow at 11am PST and you can still get streaming tickets if you’d like to tune in: http://2012.sf.wordcamp.org/tickets/

     
  • Andrew Nacin 3:33 pm on May 21, 2012 Permalink
    Tags: ,   

    On Saturday, Matt posted to the WordPress Blog that the plugin directory has been refreshed.

    Also also posted to our new P2 at make.wordpress.org/plugins: what this means for developers. @otto42, @coffee2code, and I go through the changes in detail. They include:

    • The new Developers and Support tabs for plugins
    • Subscribing to commit emails (Hint: see the Developers tab)
    • Following and managing support threads
    • How the new support statics are calculated

    (Sidebar: We hope to use make.wordpress.org/plugins for announcements and resources for plugin developers. This blog will also move to make.wordpress.org soon. More to come.)

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel