WordPress.org

Ready to get started?Download WordPress

Make WordPress Core

Updates from Denis de Bernardy Toggle Comment Threads | Keyboard Shortcuts

  • Denis de Bernardy 4:56 pm on March 2, 2010 Permalink
    Tags:   

    I cooked up a patch that allows to setup a multisite installation on localhost, albeit only for subfolder installations.

    A possibly related niggle that I ran into while installing a test site was #12459. If your database prefix is empty as well, you’ll need that patch as well.

    The primary purpose of these two is, obviously, to allow developers to use the MS functionality without needing to constantly go back and forth with a server using SSH or FTP.

    Please give both patches some testing so we can get both checked in before we’re in beta. Doing so will give the new MS functionality a lot more exposure to testers.

     
    • DD32 9:48 pm on March 2, 2010 Permalink | Log in to Reply

      Just for the rest of people, The way i’ve setup MS is to add to my hosts file a few domains.

      For reference, i’ve chosen to remap ho.st, http://www.ho.st, a.ho.st, and b.ho.st to 127.0.0.1, optionally combine that with a VirtualHost and you’ve got a MS install on localhost.

    • Andrew Nacin 9:07 pm on March 9, 2010 Permalink | Log in to Reply

      You can now use localhost for multisite subdirectory installs.

  • Denis de Bernardy 6:42 pm on January 24, 2010 Permalink
    Tags:   

    Taking advantage of UUIDs 

    Now that we require MySQL 4.1.2, we can look into using UUIDs in WordPress.

    Universally unique identifiers are numbers that are unique in time and space. In the database world, they’re primarily used in order to work with data that is created in a decentralized manner. It allows to characterize a piece of data that isn’t stored in the database yet, to merge data from separate databases without needing to work around ID conflicts, etc.

    The lack of genuine UUIDs introduces all sorts of quirks in the WP editor when we’re dealing with yet to be saved posts. #9471, #10456, #11145, #11990, and many more underscore the architectural problem that we’re facing. The media-related tickets exacerbate the issue.

    Mark recently opened #11889 and suggested that we a) always create a new post when hitting the new post button, and b) have a garbage collector clean things up periodically. It is a solution, but I feel that using UUIDs — which were designed to deal with this kind of stuff — would be simpler.

    Using them would mean pre-assigning a UUID to new posts, and placing it in a hidden field. Since this UUID characterizes our yet to be saved post, we can safely pass it around to an autosave handler, a media thickbox, etc., allowing to create the post on the fly when necessary.

    Further down the stream, we introduce a slight change in our insert/update post handler: when saving a post whose ID is not assigned yet, we start by verifying if that post exists already — by querying the posts table’s post_guid field to know if that UUID exists.

     
    • scribu 7:25 pm on January 24, 2010 Permalink | Log in to Reply

      You’ve been pushing for this aproach for some time now, Denis.

      Since you’ve got the best grip on your ideea, how about a patch?

    • Oncle Tom 7:34 pm on January 24, 2010 Permalink | Log in to Reply

      Is this a good idea to rely on such a database specific feature? How will it works if, one day, a database abstraction is used, to support other DB engines?

      It should be code logic no?

    • Matt 12:52 am on January 25, 2010 Permalink | Log in to Reply

      Philosophically, I’m a fan of fixing things with the smallest possible change that works, because it decreases the chance that an architecture change will introduce new and impossible-to-anticipate bugs.

      Also worth pointing out two old but classic essays on abstraction and architecture:

      http://www.joelonsoftware.com/articles/fog0000000018.html
      http://www.codinghorror.com/blog/archives/000165.html

      • Mark Jaquith 5:59 pm on January 25, 2010 Permalink | Log in to Reply

        That fix would probably be the “Create a draft whenever you visit the ‘Add New X’ screen” method. The bulk of the code will be the cleanup (change post_status on a real save, cleanup the unused items after a long enough time).

        And I suppose there is another benefit to this: it is DB agnostic.

        • Ryan 6:25 pm on January 25, 2010 Permalink | Log in to Reply

          If we keep guid as a varchar and wrap the MySQL syntax in a short-circuitable function we can remain DB agnostic. I don’t really want a schema change to the posts table in 3.0 anyway since we want to make MU upgrades to 3.0 as easy as possible.

          Even if we introduce UUID, I kinda like auto-creating a draft. It is very simple and fairly clean. The only messy part is the GC, but that doesn’t bother me too much. Regardless, both ways seem like solid approaches that would allow us to clean up the gallery mess.

        • Brian Layman 8:49 pm on January 25, 2010 Permalink | Log in to Reply

          A big difference I see is that one causes a DB write while the other doesn’t.

          Denis made two points advantages of the UUID:
          1. It allows to characterize a piece of data that isn’t stored in the database yet,
          2. to merge data from separate databases without needing to work around ID conflicts, etc.

          And I guess that qualifies in the first one because it avoids the need to store something. I discount the second one because we are way too far down the line for ppl not to worry about ID conflicts when merging DBs, IMHO.
          (BTW can we get some cookies around here to save our name, email and site for commenting :) )

      • hakre 6:23 pm on January 28, 2010 Permalink | Log in to Reply

        An easy one would be: User clicks Add New Post, comes to a screen where it’s a need to enter the title of your new post before user can continue to the editor. Post will be created within the request where the user enters the editor.

      • hakre 1:44 pm on February 17, 2010 Permalink | Log in to Reply

        Philosophically, I’m a fan of fixing things with the smallest possible change that works, because it decreases the chance that an architecture change will introduce new and impossible-to-anticipate bugs.

        That’s really an important thing, the smallest possible change. To achieve that in the long term, this resource is valuable as well:

        http://www.joelonsoftware.com/articles/fog0000000043.html

        Especially the point that explains when it makes sense to fix bug and how to handle changes in the code.

    • Mark Jaquith 4:28 am on January 25, 2010 Permalink | Log in to Reply

      I wasn’t quite getting this at first, but now I think I see how it could work. Is this about right?:

      Instead of populating a post ID field for a new draft, you’d populate a post_uuid field. The add attachment links, thumbnail selection link, postmeta forms, etc, would all pass this UUID if a post_id isn’t yet available. Whenever something ([whatever]) is created in relationship to a UUID and a post with that UUID doesn’t exist, you create it, and give the post that UUID (and then add the [whatever] pointing to the post ID of that newly created post). Forms that are still open and reference the UUID for the the newly created post will continue to work, except the UUID lookup will succeed, and they’ll attach [whatever] to the post ID of the post row with that UUID.

      We’ll need a function that accepts a UUID and returns either the existing post object, or if-not-exists, creates one and returns the new object. get_post_by_uuid( $uuid ) or something.

      The advantage is that we’re not creating post rows that add overhead and need cleaning up. We’d have to write code to handle the “did we get a UUID or a post ID?” stuff, but it’d probably allow us to fix all those bugs and get rid of a lot of hairy JS code.

      Denis, can you affirm or deny my stated understanding?

      • hakre 12:38 pm on January 25, 2010 Permalink | Log in to Reply

        I understand it this way: The UUID names post that is created / refered to by a user doing a specific post related action to a possibly non-existing-in-database post.

        The UUID is used to describe the simple fact, that a user knows about a post first compared to our current post-related-data-structure(s) in core. I like that Idea.

      • Denis de Bernardy 3:12 am on February 5, 2010 Permalink | Log in to Reply

        “Whenever something ([whatever]) is created in relationship to a UUID and a post with that UUID doesn’t exist, you create it, and give the post that UUID (and then add the [whatever] pointing to the post ID of that newly created post). Forms that are still open and reference the UUID for the the newly created post will continue to work, except the UUID lookup will succeed, and they’ll attach [whatever] to the post ID of the post row with that UUID.”

        sorry I didn’t reply earlier (sick, then busy), but yes, that pretty much is exactly it…

    • miqrogroove 1:54 pm on January 25, 2010 Permalink | Log in to Reply

      A third and very simple strategy would be to have the front end keep track of the image IDs whenever there is no parent ID, and then include that ID list with the first auto save. That way you have no GC, no schema change, and no empty drafts..

    • Brian Layman 5:30 pm on January 25, 2010 Permalink | Log in to Reply

      Coming from a database background where 50gb dbs were not that unusual, I still cringe thinking at PHP’s loose typing and handling numbers as strings. Reading up on UUIDs it is neat to see that it is a actually stored as a 128bit integer. Which I thought was a really good thing, but because this will be passed around as a string, this is not the end of the story:

      There’s a warning in the documentation that character sets play a significant role in UUIDs. I’m not sure how that will affect us, but you can setup a case where the character set of the column doesn’t match the values stored in that column and therefore the indexes fail.

      Maybe this can be fixed by regenerating the index, I don’t know. Is it a concern? It might be if the blog is set to a different character set than the database right? I’ve run into problems like that before when the data was a different character set than the structure in which it was stored. Maybe it was only because we do a lot of tossing around of blogs…

    • Joseph Scott 6:17 pm on January 25, 2010 Permalink | Log in to Reply

      Which type of UUID was being considered for this?

    • Brian Layman 7:14 pm on January 25, 2010 Permalink | Log in to Reply

      I don’t think the MySQL UUID() function has various types.. http://dev.mysql.com/doc/refman/5.0/en/miscellaneous-functions.html#function_uuid

      • Joseph Scott 8:30 pm on January 25, 2010 Permalink | Log in to Reply

        From the discussion so far it seemed implied that the UUID would be generated in PHP, not MySQL. That said it looks like the UUID showed up in MySQL 4.1.2 so we could potentially use it.

    • Peter Westwood 9:49 pm on January 25, 2010 Permalink | Log in to Reply

      I am really against trying to use UUIDs as the linkage between posts and there metadata/attachments.

      We have a perfectly good method of providing this linkage and a simple solution has been proposed to resolve the issue with autosave prior to a draft existing.

      All we need is for get_default_post_to_edit() to ensure something exists in the database and most of the rest of the code won’t need touching.

      I am with Matt on the small change is better.

      UUIDs aren’t designed to solve the issue we have here – they are designed as a way of letting you generate IDs that are unique accross multiple sites/machines and are a candidate for using as the GUID in the posts table but not as the linkage/post_id

      • wpmuguru 5:58 pm on January 26, 2010 Permalink | Log in to Reply

        +1
        The problems being addressed in those tickets occur infrequently. Reinventing the post ID management system seems to be an excessive solution to the problem.

      • hakre 5:02 pm on January 28, 2010 Permalink | Log in to Reply

        Westi, from what you write in your comment is that you did get it wrong. The UUID would not be used to replace the Post ID, it would be only in use as long as there ain’t no Post ID. You understand the difference?

        • Peter Westwood 5:50 pm on January 28, 2010 Permalink | Log in to Reply

          Yes. But it would effectively replace post ID.

          Either we would have to have all the code cope with matching based on ID or UUID or we would have to switch solely to using UUID.

          Introducing UUID for posts that aren’t saved yet just introduces unnecessary complexity

        • hakre 6:11 pm on January 28, 2010 Permalink | Log in to Reply

          The unnecessary (?) complexity was introduced by users who do upload attachments for non-existing posts because the UI made them think, they are actually uploading images to a new post.

          The UUID is only a technical thingy to make the software again capable of that situation. The UUID itself is not introducing any more complexity. The opposite is the case, it makes things more simple and useable. The current data structure is not able to cope with the situation we have in the post editor.

          Or what do you suggest as analysis why this bug is staying open so long unfixed?

        • Peter Westwood 7:52 pm on January 28, 2010 Permalink | Log in to Reply

          If we introduce a UUID just for this situation we have to write a lot of extra code for WordPress to handle the action on an un-saved post to handle the action and every plugin author that wants to do something similar has to do this too.

          If we just always create a draft then the current code in core and plugins will just work.

    • Brian Layman 5:13 pm on January 28, 2010 Permalink | Log in to Reply

      I wasn’t going to reply because comments on this had slowed, but I think I got it wrong too. I don’t think that Denis implied any changes at all to the data structure. If you read the last paragraph, he is envisioning the UUID being stored in the post_guid field. So his proposal is one for the editing screens only and not an altering of the post table’s structure. If I understood it more correctly upon re-read that is.

      • Peter Westwood 5:51 pm on January 28, 2010 Permalink | Log in to Reply

        This would still affect a lot of code which wouldn’t need changing if we just created a draft before outputting the page.

    • Jeremy Stark 5:53 pm on February 19, 2010 Permalink | Log in to Reply

      I wrote a UUID plugin in order to integrate WordPress with another publishing system:

      http://wordpress.org/extend/plugins/simple-uuid/installation/

      It just adds a uuid to each posts metadata.

  • Denis de Bernardy 2:48 pm on January 23, 2010 Permalink
    Tags: ,   

    I’m inaugurating a new trac keyword, “bug-hunt”, and the bug hunt report, in order to keep track of of tickets that get posted/could get posted in this blog’s “bug hunt” posts.

    Updated: I’m inaugurating a new trac keyword, “featured”, and the featured bugs report, in order to keep track of of tickets that get posted/could get posted in this blog’s “featured bugs” posts.

    I’ll be tagging more tickets in due course. You’re most welcome to mark additional tickets as such, if you feel they qualify.

    For information, I try to stick to batches of tickets that loosely interfere or interact with one another. The batch of tickets should then meet the two or more of following criteria:

    • It is annoying when you run into the issue
    • It underscores an architectural problem
    • Fixing it can lead to performance optimizations

    Areas I plan to highlight in the future include: UUIDs, i18n post slugs, permalinks, rewrite rules, cache, private posts.

    Btw, I’d like to stress that the point in these blog posts is try to reverse the WP bug trend:

    Open WP bugs, props Hakre

    Thus, I’d like to revisit two of my last post’s tickets:

    1. #10779, on optimizing our unzip method. DD32 committed the fix, but marked it as still needing testing. Here’s how. Related ticket #10403 is still pending, but I feel it’s extremely minor compared to #10779.
    2. #10913, on optimizing the upgrade process, is still a major niggler imo. #10611 will probably seem cosmetic if #10913 gets fixed. Two possible solutions are highlighted in the ticket’s comments. It still needs a patch, and of course testing once one is around.

    We need your help on both, and on future tickets that will make it in these posts. Thanks for patching/testing.

     
    • Ryan 6:59 pm on January 23, 2010 Permalink | Log in to Reply

      It would be interesting to see the trend for enhancements/feature requests. Currently, 53% of open tickets are enhancements and feature requests.

    • DD32 10:54 pm on January 23, 2010 Permalink | Log in to Reply

      I’m wondering if ‘bug hunt’ is the correct term for these.
      Historically Bug Hunt days are testing a heap of bugs and/or patching them..
      The criteria hardly relates to what Bug Hunts used to be (not to speak of them in ast tense, but its been a little while). So i’m wondering if its going to confuse some people, Its not a bug hunt, its just tickets which you see as annoying, or a possible optimization – Many of the latter is not something most people can even attempt to look at.

      For a start, #10611 is neither a bug, nor does it have a patch, So whilst it’s a performance thing, Its not something that most people should be told to test..

      As for #10779 however, I need to know who its failing for, and what kind of server setup they have.

      I’d like to see Bug Hunts left to things that end-users can attack and test, and make sure works for them before being commited (Or testing the commited code still works for them)

      • Denis de Bernardy 3:44 pm on January 24, 2010 Permalink | Log in to Reply

        Fair enough. Would you find the keyword “featured-bug” more appropriate?

      • Denis de Bernardy 4:13 pm on January 24, 2010 Permalink | Log in to Reply

        I changed it to “featured”

        • Jane Wells 2:54 pm on January 25, 2010 Permalink | Log in to Reply

          The word Featured has a lot of connotations in terms of official endorsement and promotion (featured plugins, etc), which we currently already have in trac with the keyword “blessed.” If the point of the tag is that you are asking for the bug to be given a higher priority, a more appropriate tag would be “denis” or “please-look” or something that is more specific.

    • Andrew Ozz 12:23 pm on January 28, 2010 Permalink | Log in to Reply

      Looking at the chart above: there were a lot of fixed bugs/closed tickets for 2.8 and at the same time it had more problems than either 2.7 or 2.9. So it seems “more tickets closed” equals more problems at release.

      Going through the currently open tickets with patches marked as bugs, lots of them introduce some sort of architectural change (no matter how small) which eventually breaks something else. Even tickets marked as “tested, commit” have usually been tested only for the problem described in the current ticket and potentially introduce new bugs.

      In that terms I agree with DD32: a “bug hunt” should be a lot less of “patch a bug” and more of “test the patch for a known bug” and leave a comment on the ticket about what was tested and how. Having a list of patches to start with may work but generally most patches need this type of testing.

    • Matt 6:15 pm on January 28, 2010 Permalink | Log in to Reply

      Looking at the chart above: there were a lot of fixed bugs/closed tickets for 2.8 and at the same time it had more problems than either 2.7 or 2.9. So it seems “more tickets closed” equals more problems at release.

      I am in violent agreement with this comment.

      • hakre 6:34 pm on January 28, 2010 Permalink | Log in to Reply

        Take note, that the graph does not differ between actual bugs and features as Ryan already correctly pointed to. So Andrew Ozz sentence you violently agree with could be written this way as well:

        “So it seems “more features implemented” equals more problems at release.”

        Just to show the picture.

  • Denis de Bernardy 12:57 am on January 13, 2010 Permalink
    Tags: ,   

    So… Jane had asked me to come up with a post now and then that discusses bug squashing. Between the Health Check plugin, chewing on the best way to do this, and noting that our lead devs were quite busy already because of the merge and custom post types, I had yet to start.

    The opportunity comes courtesy of Matt’s latest wp-devel post. So here goes… If you’re wondering how you might be able to make the best of DD32′s new contributor status, here’s a selection of WordPress niggles that could use patches, second opinions, and of course lots of testing:

    1. #11588 is about actually showing a message to end users during core upgrades. There are occurrences where the host enables an output buffer automatically. When this happens, end users a greeted with a partially loaded screen that can seem to wait forever. Status: ideas submitted for force-flushing, and needs-patch.

    2. #10779 is a huge source of grief on overcrowded, low-end servers. Our unzip method is slow and resource hungry, and could use some optimizations. (See also #10403 on the same topic.) Status: needs-testing.

    3. Another possible optimizations for these same servers would be #10611. The idea would be to only upgrade the files that changed, when upgrading minor releases. Status: needs-patch.

    4. #10913 highlights the reason why failed upgrades are such a source of grief. When using the FTP transport, file stats don’t get cached, and so we end up needing to re-check the status of each of them constantly. On the slower servers, the optional clean up + unzip + upload + clean up procedure frequently fails because of it. The whole process could be optimized by caching $fs::is_file() and $fs::is_dir(). Status: needs-patch.

    5. #8830 loosely relates to the filesystem, but definitely seems to cause grief for users with an open_basedir restriction in effect. From the looks of it, a simple rtrim() call could fix wp_mkdir_p() on those system. Status: needs-testing.

    6. #10889 highlights inconsistent methods between the WP Filesystem methods. I’m 90% sure that it prevents the .maintenance file from being added on some sites. Status: commit?

    And that’ll be the last of today’s pick of crippling install/upgrade bugs. Please keep in mind that this is the first of a hopefully long series of posts. Feedback is welcome on the way it gets presented. Should it be more focused on less specific bugs, less focused but on more bugs, less focused on less bugs, or is it reasonably balanced as above?

    Next in line, I think, will be a post that discusses the use of UUIDs to fix a couple of tickets, along the lines of #11145. Now that we’ve MySQL 4.1.2, we might as well take advantage of them.

     
    • miqrogroove 1:34 am on January 13, 2010 Permalink | Log in to Reply

      I’m looking forward to your next post because there’s a reliable cure for remote upgrades: manual upgrades. On the other hand, bugs like #11145 are a disinsentive to upgrade at all. Fixing them also has the benefit of making patches that usually work on older versions. That’s just as important for those of us who absolutely cannot upgrade to the 2.9 branch until plugin authors catch up with the core devs.

    • miqrogroove 1:35 am on January 13, 2010 Permalink | Log in to Reply

      Good lord, what has happened to my avatar….

      • miqrogroove 2:48 am on January 13, 2010 Permalink | Log in to Reply

        Apparently wordpress.org, wordpress.com and gravatar.com were using 3 different e-mail addresses for me. I think it’s straight now. Back to sunny Robert again.

    • Ryan 7:29 am on January 13, 2010 Permalink | Log in to Reply

      This is entirely off-topic, but just wanted to mention that it’s awesome you guys are allowed to post on here. It’s making it a lot easier to track what’s going on behind the scenes at WordPress.org.

      • Matt 7:30 am on January 13, 2010 Permalink | Log in to Reply

        Happy to open this up more, if someone wants posting ability just ask, and link to your WP.org profile showing your Trac contributions.

    • Andrea_r 1:19 pm on January 13, 2010 Permalink | Log in to Reply

      This is actually presented quite nicely in a reasonable chunk of related items. I like it.

    • Ryan McCue 2:23 pm on January 13, 2010 Permalink | Log in to Reply

      I have to say, I love the line “FWIW, the suggested patch doesn’t flush hard enough.” :)

    • Jane Wells 3:18 pm on January 13, 2010 Permalink | Log in to Reply

      My only suggestion would be to limit it to no more than 3 per post, in order to keep it short enough that it feels like it will only take a minute to open them and respond. Thanks for doing this!

  • Denis de Bernardy 4:36 pm on January 8, 2010 Permalink  

    Collecting ideas for the Health Check plugin 

    I’ve started to give some love to the Health Check plugin. Quite a few PHP-related tests have been added already. Peter has a few more ideas in stock, and I plan to quickly add some MySQL checks.

    We could use some additional ideas. I’ve opened a forum thread accordingly. Basically, if you’ve a grudge against this or that server config, or if you regularly help users who experience the same issue over and over, be sure to speak out, either in here or in the forum.

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel