Tag Archives: weekly-update

Kat Hagan 7:08 am on July 19, 2013
Tags: , weekly-update   

Post By Email Update: POP/IMAP libraries, Trac questions

Rather unexciting update this week, as I’m still in the midst of a few things (and have been distracted by caring for a cat who had some teeth pulled… poor kitty!).

As I mentioned last week, in reading through old TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. tickets, I learned that the POP3 class we’re using to read emails was originally copied over from SquirrelMail.  I’ve been digging into their SVNSVN Subversion, the popular version control system (VCS) by the Apache project, used by WordPress to manage changes to its codebase. repo in hopes of being able to adopt (1) an updated POP library that might include fixes for some of the bugs we’ve found, and (2) an IMAP library that we can similarly adapt to our own uses.

It’s an interesting adventure trying to reconstruct years of history from bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. tracker and commit logs… at any rate, it appears the POP3 library originally came from a SquirrelMail pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party called “mail_fetch”, which was since merged into SquirrelMail coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress..  Complicating matters, the file was renamed from class-pop3.php to class.mail_fetch.php.  But I did ultimately track it down!

However, while the development branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". of SquirrelMail now has “some plumbing” to support remote IMAP servers, I’m not sure whether any of that code will be useful for our needs — it’s not just still under development, but much more intertwined with the rest of SquirrelMail’s code, as opposed to being a standalone class.  At this point I’m more inclined to go with a different library, and have been researching alternatives; the top contender right now is the Horde IMAP library, which also includes an abstraction layer for POP3 (but I’m open to suggestions, if anyone has a favorite).  In the next week, I plan to load that into the plugin and see if I can get it working to support a wider variety of mail servers.

In other news, I’ve found myself fighting with Trac a lot lately.  I’m curious how all y’all keep track of tickets — both existing ones that you want to receive notices about, and new ones coming in that might be relevant to your interests.  Do you have custom searches bookmarked, an RSS feedRSS Feed RSS is an acronym for Real Simple Syndication which is a type of web feed which allows users to access updates to online content in a standardized, computer-readable format. This is the feed., something else?  Please share any tips!

#post-by-email, #weekly-update

Alex 8:09 pm on July 16, 2013
Tags: , weekly-update   

Code Revisions: Week 4

Version 0.4 of the Code RevisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party is out. If you used a prior version you might need to manually remove posts of the post type code and post metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. fields with the key _code_revisions. Uninstall and upgrade functionailty which removes/refreshes this stuff automatially will of course be integraded in later versions.

Main new feature this week: Taking direct file changes through ftp or plugin/theme updates into account (#303). A new revision is automatically created and the user is notified with an adminadmin (and super admin) notice:

This file changed in the meantime.Not sure about the text there yet 😉 But this can be changed later on. A problem I ran into here was that I need to approach this from a plugin point of view and cannot hook into coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. in all places I would possibly like to. The intial idea was to save a file’s last modified timestamp (using filemtime) and check it when ever viewing the file in the editor again. Problem is the file is written right before the page is redirected – which is why I cannot get the file modified time after the file was written. So now I am using md5-checksums of the file’s content instead. This results in an additional file read to generate the checksum (using md5_file) every time an editor is being viewed – again, this could be avoided if I could check the sum after the file is read for being used in the editor. If this functionality gets into core I will need to move to one of the more direct approaches.

Another thing which did cost me quite some time to find out: The time points revisions are created by core changed. I mostly try to stick to core functionality and for creating revisions only used wp_insert_post and wp_update_post – revisions were created automatically. Now this behavior was changed for wp_insert_post (#24708) and I did need to adjust a little bit (#324). This now results in a new problem because the first two revisions are created in the same second and therefore considered to both be the currently “live” revision (#326). Most simplest way might be to apply some JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors. to reactivate the button. Need to look into this.

Midterm and with it “feature completeness” is moving closer. Next on my list is to prevent user’s from breaking their installation through fatal errors.

See you next week, I would appreciate testing and maybe some bug reports! Thanks!

#code-revisions, #weekly-update

Frederick Ding 12:00 am on July 16, 2013
Tags: , weekly-update   

Migration update: WXR importer

This week, I began work on the next phase of my project: fixing up the WXR importer plugin. A number of developers, including Jon Cave, Peter Westwood, and Andrew Nacin have been maintaining this pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party since at least May 2010.

I forked the code from the plugins repository into my GSoC Subversion directory in preparation. It’s taken a while to test this (manually) against XML files from existing sites, so that I can see under what circumstances it fails to complete the import or perform to expectations, and what can be done. Trac tickets and forum posts have been informative as well. (See the linked posts for the results and observations.)

I’ve also run the unit tests that apply to the importer plugin; however, the test cases are generally small (indeed, the biggest XML test case is 26 KB, titled small-export.xml) and don’t trigger the kinds of issues that importing dozens or hundreds of posts and attachments—with WXR files of megabytes in size—does.

So, the first task at hand is breaking up the process—which currently executes in one step with little indication of progress—into discrete chunks that can run in separate, stateful (stepwise) requests.

A chat with my mentors has pointed me in the direction of WP_Importer_Cron, which was first developed for other importers that need to make external APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. calls (e.g. Tumblr Importer) potentially subject to rate constraints. There are some parallels between “external API calls” and “remote attachment fetching”, which is why this can be a suitable approach for fixing the timeout issues that present with the current WordPress importer. After the process is discretized, showing progress (an enhancement long overdue) will be easier.

#migration-portability, #weekly-update

Kat Hagan 9:04 am on July 12, 2013
Tags: , weekly-update   

Post By Email Weekly Update

This past week, I continued delving into the mysteries of unit testing, and ended up submitting a minor patch back to the unit-tests repo (after spending some time trying to figure out what was wrong with my setup that was causing one of the tests to fail on coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.).

The Post By Email pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party now has working tests for activation, but not yet for the main functionality, which will require further research to learn how to mock out email retrieval.  (I’ve back-burnered this for the moment.)

I also spent some time wandering around TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress., and learned that a problem I’ve been having is actually an ancient bug, which I then tracked down to a misbehaving regex that is supposed to determine the charset of the email.  It requests that iconv transform the given text into the garbled charset, gets nothing back, and proceeds to cheerfully create an empty post.  I haven’t fixed it yet, as I’m hoping to replace large swaths of that code with a new library anyhow (apparently we originally got that POP3 library from SquirrelMail, which does now support IMAP and SSLSSL Secure Sockets Layer. Provides a secure means of sending data over the internet. Used for authenticated and private actions., so that’s the first place I’ll look unless there are other recommendations).

Finally, I collated the plugin’s status and error messages into a mini-log, and added a quick status overview to the settings page.  I also put a “check now” button in the settings.  Right now it just logs the result of the most recent check; ultimately I plan to show a longer history, and also organize/format the messages better (errors color-coded, links to posts, etc.).

I tweaked a few more things about my setup this week, and am now mirroring the plugin source on Github, if GitGit Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. Git is easy to learn and has a tiny footprint with lightning fast performance. Most modern plugin and theme development is being done with this version control system. https://git-scm.com/. happens to be more your cup of tea.

Next, I’m focusing on developing a patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing. to deprecate Core, and vetting existing plugins to ensure backward compatibility.

#post-by-email, #weekly-update

Alex 4:59 pm on July 9, 2013
Tags: , weekly-update   

Code Revisions: Week 3

Week 3 of GSoC: Half-way to midterm and half-way to feature complete betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process..

The last week was about restoring code revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. (#296). There again the problamatic part was how differently WordPress handles themes (#306) and plugins (#307). By now I am actually considering to streamline the two editor files (theme-editor.php and plugin-editor.php) using a general code editor class. Espacially pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party-editor.php does some weird stuff: The plugin query var for example often contains the last viewed file instead of the plugin’s main file. There are actually already multiple trac tickets regarding stuff like this. But this would be out of scope of my project – maybe something for after midterm, depending on the project’s progress.

Until next tuesday I will teach the plugin to pay attention to direct file changes through ftp or theme/plugin updates (#303). Users should be able to see how their own changes were overwritten. This is of special interest when the user intends to reimplement his own changes or wants to check if bugs he fixed are now fixed officially.

If you are interested you can test version 0.3, report bugs and follow on further progress on trac.

#code-revisions, #weekly-update

Frederick Ding 12:57 am on July 9, 2013
Tags: , weekly-update   

Migration update: WP Relocate

This past week, I’ve focused primarily on bringing a usable interface to the WP_Relocate class that I posted about last week. While the WP_Relocate class file itself was meant to be something that could be bundled into another suite and reused, the fruits of this week’s labour come in the form of an installable pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party.

In actuality, the plugin hooksHooks In WordPress theme and development, hooks are functions that can be applied to an action or a Filter in WordPress. Actions are functions performed when a certain event occurs in WordPress. Filters allow you to modify certain functions. Arguments used to hook both filters and actions look the same. are very shallow — merely used to add a menu link (for now). The most important part is the practically-standalone interface.

WP Relocate UI

The design intent was to make this look and feel just like the installer — or the upgrade.php script for database schema changes.

This has been tested against 3.5.2 and 3.6 only. Despite differences in how these versions handle revisions, the changes made to post content are still revertible in either version:

Revisions tracking search and replace

I can test this in code to my heart’s content (except for the difficulties using the current unit-tests framework against older versions of coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.) but nothing beats running it on live site data, especially sites that have posts and uploads from earlier versions of WordPress. As you might imagine, live data is more difficult to generate and test against. I don’t yet know how well this process works with hundreds or thousands of posts, in terms of the time it takes and the level of verbosity communicated to the site administrator.

If you’re interested in trying this out (please don’t use it on a production siteProduction Site A production site is a live site online meant to be viewed by your visitors, as opposed to a site that is staged for development or testing. — it’s probably not ready for that), feel free to:

It installs into wp-content/plugins. The included readme.html file contains more detailed instructions. If anyone tries it out, I’d love to hear if it broke your site. 🙂

What’s next? This week, I am forking the importer plugin that works with WXR files, to examine and fix its issues with fetching attachments from the source. I hope to add some ability to replace URLs using the WP_Relocate class to smooth out the process of copying content from another installation.

Edit (2013-07-09): updated to 1.0.1 with fix for PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher versions before 5.4.0.

#migration-portability, #weekly-update

Kat Hagan 6:48 am on July 5, 2013
Tags: , weekly-update   

Post By Email update: tests and multiple WP versions

First off: After I mentioned last week that I needed a POP mail account to test with, several folks with their own mail servers offered their help.  I’m set up with something now.  Thanks to all who reached out!

The past week has been all about testing, with the goal of getting good unit and/or integration tests written for the Post By Email pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party before I start adding features to it.  This is entirely new territory for me, though I’ve done some TDD in Ruby in the past.  So far, I’ve learned how the tests work on CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., and gotten a whole mess of things installed and running (wp-cliWP-CLI WP-CLI is the Command Line Interface for WordPress, used to do administrative and development tasks in a programmatic way. The project page is http://wp-cli.org/ https://make.wordpress.org/cli/, Pear, PHPUnit, etc.).  I followed this guide to set up the basic test structure for the plugin.  I was hoping to have some actual tests done for this week’s update, but it took a bit longer than expected to get the framework set up, so I don’t have anything ready for primetime yet.

So, this week’s question: What are some plugins that have really good tests?  I’m seeking examples of what to do (or not to do!).  I’m especially curious to see tests of Settings APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. stuff and install/activation functions.

Side note: I really need to clean up my development setup for testing against multiple WordPress versions.  I have several versions installed already, but they’re pretty haphazard (with different databases, inconsistent directory names, etc.).  I found this very good post about a development setup with multiple WP versions.  For those of you who support plugins or themes on multiple versions — any tips?  Is your setup similar to that post?

#post-by-email, #weekly-update

Alex 4:58 pm on July 2, 2013
Tags: , weekly-update   

Code Revisions: Week 2

What’s going on here? Thought tuesday is code revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. day here on make coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress... 😉

Well, doesn’t matter that much, not much to report today. Version 0.2 gives you a metaboxMetabox A post metabox is a draggable box shown on the post editing screen. Its purpose is to allow the user to select or enter information in addition to the main post content. This information should be related to the post in some way. below the code editors listing all revisions as you know it from the post and page editor (#286). The metabox tries to replicate the original as close as possible. A difference here is that it is created using JSJS JavaScript, a web scripting language typically executed in the browser. Often used for advanced user interfaces and behaviors. and an AJAX request. This ensures that I don’t have to edit core files but it also results in no metabox with JS disabled. As it looks at the moment the latter won’t be a problem because revision viewing will be JS-only starting with 3.6. Furthermore from now on you get redirected to the corresponding code editor when trying to access the post editor for posts of the code custom post typeCustom Post Type WordPress can hold and display many different types of content. A single item of such a content is generally called a post, although post is also a specific post type. Custom Post Types gives your site the ability to have templated posts, to simplify the concept. (#289). This is especially helpful when clicking the title displayed at the top of wp-admin/revisions.php

At the moment I am closely following the final pre 3.6 changes in the revisions component. The next steps for the project is to introduce restoring (#296) and to pay attention to direct file changes (#303).

#code-revisions, #weekly-update

Frederick Ding 10:19 pm on July 1, 2013
Tags: , weekly-update   

Migration project update

I’ve spent the past few days building functionality for what I’ve termed the “relocate” component — the class that handles site-wide replacement of old URLs with new. A pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party UIUI User interface is in the work.

Two discoveries were made in the process of coding the relocate component.

  1. Attachment locations have been stored as relative paths since 2.7, which helps with keeping them portable. However, I have seen full paths in the database tables of sites that started up pre-2.7; transitioning those to portable relative paths will be part of what I’m building.
  2. By using WordPress queries and post functions (as opposed to SQL queries) to replace within post content, we gain the advantage of storing those changes in revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision., rather than indiscriminately touching all entries in the posts table in the way that was previously documented on the Codex.

With the guidance of WP CLI’s explanation for how to set up a plugin for testing, I’ve also coded unit tests to verify that things are working as they should. One hurdle I faced in unit testing was that update_option('siteurl') doesn’t affect the value of the WP_CONTENT_URL constant until the next execution of the script, by which time the unit testunit test Code written to test a small piece of code or functionality within a larger application. Everything from themes to WordPress core have a series of unit tests. Also see regression. framework will have rolled back the transaction. This minor quirk won’t translate to real world use, although I’m not sure if I’m satisfied with the way I overcame it.

Finishing the UI for this component is my timeline’s task for this week; it will facilitate a user’s decision to change the site URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org to a different domain. I’ll make sure installation/usage instructions are shown by next week.

Happy Canada Day!

#migration-portability, #weekly-update

Kat Hagan 9:47 am on June 28, 2013
Tags: , weekly-update   

Post By Email Weekly Update

I’ve spent the past week converting the Post By Email pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party into proper OOP form, trying to incorporate the best practices for plugins as well as the WP coding standards (from which I learned a few things… like why you’d want to leave the end tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) out of PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher files!).  I cribbed a lot from Tom McFarlin’s WordPress Plugin Boilerplate.

On activation, the plugin now copies the global Post By Email options into its local settings and hides those settings from Settings->Writing.  There’s also an options page that uses the Settings APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways..

Hilariously (and illustrating the dire need for an upgrade), I’m having a hard time figuring out how to test this; it turns out it’s actually quite difficult to find a free email provider that still does unsecured POP.  Suggestions?  I might have to add SSLSSL Secure Sockets Layer. Provides a secure means of sending data over the internet. Used for authenticated and private actions. support just to be able to retrieve any emails.

Anyhow, I’m calling this version 0.9, with 1.0 being all the functionality from coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. (nothing extra yet).  Once I upgrade it to a full 1.0 version, I’ll post installation instructions here so the curious can try it out and tell me what breaks.

Next steps:

  • Show log info on plugin options page  (e.g. “Last checked at 2:15pm, 1 message posted.”)
  • Find an email account to test with (or just add SSL?)
  • Add a “check mail now” button on the options page

#post-by-email, #weekly-update