Minor programming notes

Quick announcement before things totally shut down for the holidays:

  • Martin Burke (@gitlost) will be taking the Release Lead role for WP-CLI v1.5.0, due out on January 30th. He’ll be making sure all of the i’s are dotted and t’s are crossed before the version is tagged and deployed.
  • Alain will continue on as maintainer in a supporting role to Martin, helping out on a day to day basis with triaging the deluge of issues and pull requests.
  • I (Daniel) will be stepping away from day to day maintenance of WP-CLI, but will be around to support Martin and Alain as needed.

But why? Is this the end of the world?

My goal is, and always has been, long-term stability and quality for WP-CLI. I want WP-CLI to be software you can depend on for years, not months.

Part of longevity is consistent baseline investment into maintenance, with occasional bursts of new feature development. But longevity is also the human capacity around the project, of which I cannot be the bus factor. Stepping away creates space for others to step up.

Martin and Alain are very intelligent individuals. I trust they’ll do a great job.

Are you planning to come back?

We’re trialing this through the release of v1.5.0, and will reassess after that point.

I’m personally looking forward to: 1) taking a true mental break (my first WP-CLI PR was September 2012), and 2) submitting pull requests to WP-CLI as a contributor again.


If you’ve been hesitant to contribute to WP-CLI, now’s your time to dive in and help out. I’m sure Martin and Alain would love your enthusiasm 🙂

Live long and prosper!

Call for action – Help us test Checksum Verification

We’ve been working on building a first usable implementation of the plugin checksum verification project. Now we need your help to test the current implementation.

Implementation Details

The WordPress.org infrastructure now calculates MD5 and SHA-256 checksums for all plugin files and stores them in a publically accessible way. You can find a specification of the current endpoint to retrieve the checksums here.

The wp checksum plugin command we’ve built goes through some or all of the plugins installed on a machine, downloads the checksums for each plugin, and then verifies the downloaded checksums against freshly generated ones.

We now need help testing this command to make sure we weed out all edge cases and that its output serves all expected scripting requirements.

Right now, the output on STDOUT will provide you with a list of checksum mismatches or added/removed files. STDERR will contain warnings about skipped plugins. The exit code will return 0 if all compared checksums were valid, and 1 otherwise. Any feedback on whether that is a good approach, or on alternative approaches for the output are welcome!

Let us know as well when a plugin’s checksums is not found that you would expect to be found in the official plugin repository. Note: Right now, only the checksums for the latest versions of every plugin have been calculated, older versions will be added later.

How To Test

The implemented command can be found in the plugin-checksums branch of the wp-cli/checksum-command repository.

You can easily install the version to test through the following command:

wp package install wp-cli/checksum-command:dev-plugin-checksums

To get back to the stable bundled command later on, just type the following command:

wp package uninstall wp-cli/checksum-command

The easiest way to run the test is to enter the root folder of an existing WordPress site and run the following command:

wp checksum plugin --all

The command supports several formats, like JSON or CSV, which you can generate through the --format=<format> parameter.

Note: the output will be most useful right now if all plugins are up-to-date (as older checksums have not been calculated yet), so you might want to run a wp plugin update --all against local sites you test. Obviously, don’t do this without backups on production sites.

Please report any feedback or issues you find in the GitHub issue tracker of the checksum command.

Timing for v1.5.0 and v2.0.0

A heads up on some upcoming scheduled changes:

  • WP-CLI v1.5.0 is slated to be released on Tuesday, January 30th. This will be the last release in the v1.x series, unless some critical issue necessitates a v1.5.1.
  • WP-CLI v2.x will require PHP 5.4 and higher. Given current release cadence, v2.0.0 could be released in the May timeframe.

Why? Travis CI is ending support of PHP 5.3 testing (targeting early April). I’m not very keen to invest time into some alternate CI system solely for those still running PHP 5.3.

If you manage WordPress installs for others and don’t already have systems for helping them safely update PHP, you should take the time in 2018 to develop said systems.

What do you wish `wp scaffold` could do for you?

wp scaffold * presents a series of commands to help you quickly get up and running with your development project:

  • wp scaffold plugin generates a basic plugin, complete with unit tests.
  • wp scaffold post-type produces a full custom post type, with all of the different arguments you can change.
  • etc.

We’d like to make it even better! To help guide your feedback:

  • What basic code do you find yourself copy and pasting between WordPress projects?
  • Of the existing commands, are there ways you’d like to improve upon them?
  • What boilerplate/scaffolded code do you find most helpful in the other open source projects you use?

Consider this an open thread: we welcome all the ideas you have.

WordPress Plugin and Theme Checksums Project – Announcement

Overview

WP-CLI provides a way for system administrators to verify the integrity of the WordPress core files. Through wp checksum core, you can easily verify that a given installation has not been tampered with. It not only checks whether the correct files are in place, but also that their content has not been changed. This is possible because WordPress provides an official API to check the expected core file checksums at https://api.wordpress.org/core/checksums/.

Having this kind of functionality for plugins and themes as well would be a huge security benefit. It would allow you to check the file integrity of an entire site, possibly in an automated fashion. However, there is no centralized way of retrieving the file checksums for plugins or themes yet, and the alternative of downloading the plugins and themes from the official servers first just to check against them is wasteful in terms of resources and bandwidth.

The aim of this project is to extend the checksum verification and its underlying infrastructure so that it can reliably and efficiently check the integrity of plugins and themes as well.

Project Stages

The project will be structured into four stages. Each stage will be followed by a detailed report, containing a summary of the stage’s efforts as well as a clear enumeration of decisions and results.

A. Initiation (← we are here)

During this initial project stage, we raise awareness of the project and discuss it with key stakeholders, sponsors, and volunteers.

We’ll evaluate the alternative approaches with all involved parties to distill the most viable path to a maintainable solution.

Finally, we’ll define a clear scope for the project, and the metrics that define its success. We plan for a working beta version by end of November, so we will want to keep the scope tight for this first iteration.

B. Planning

After we’ve decided on a specific route to follow, we can start planning the details of the solution we want to implement.

This stage will result in a project roadmap with milestones and their respective deliverables. It will also produce a list of requirements, like the provisional budget for infrastructure, the decisions needed or the estimated workload for each milestone.

C. Implementation

After we’ve planned all the technical details and broke down the work involved, we’ll start with building the infrastructure and implementing the client and server software.

The specifics of how this stage will be handled should have been laid out during the planning stage already, so this stage is all about execution and monitoring progress.

D. Integration

During the final project stage, we will move all code and infrastructure to reside under the official wordpress.org domain and complete the integration with the WordPress Core and the WP-CLI tool.

Get Involved!

This project will have a huge impact on the perceived and effective security of WordPress installations. It can greatly reduce the amount of malware-infested sites plaguing the internet, and through the substantial market share of WordPress, improve the general browsing experience for all net citizens.

If you want to get involved, you’d ideally meet the following criteria:

  • You have a vested interest in security and/or system administration in a WordPress context.
  • You can spare a consistent average of ~5+ hours/week (hopefully on your employer’s time).
  • You have experience with one or more of the types of components this project requires.

If this is you, please get in touch with us, either by commenting on this post or by joining the discussion in the following GitHub issue: https://github.com/wp-cli/ideas/issues/6.

We will have a formal kickoff during the next WP-CLI office hours in the #cli channel on October 3rd, 2017 at 16:00 UTC. Feel free to join the discussion and help us get this ball rolling.

Version 1.4.0 released

Happy release day!

We’re excited to bring you WP-CLI v1.4.0. In just two short months, we’ve merged 308 pull requests from 42 contributors.

New faces

You may have noticed a few new faces around the project. This is our motley crew of committers:

  • Martin (@gitlost) lives in Dublin, Ireland. Check out his work with wp db search and wp search-replace --log (see below).
  • Siddharth (@Sidsector9) resides in Pune, India. His enhancements to wp doctor and wp profile will be available soon in a release near you.
  • Takayuki (@miya0001) is based out of Kyoto, Japan. He’s been making numerous improvements across the entire project; you never know what his next pull request will fix.

They’ve already had an amazing impact on the project. Please pass along your thanks when you have the chance.

Log search-replace transformations

If you’ve ever wanted to see what transformations are taking place with wp search-replace, now you can!

Use wp search-replace --log to display transformations as they happen, or wp search-replace --log=transformations.log to save the transformations to a file [#35, #39]:

$ wp search-replace 'http://' 'https://' --log
wp_options.option_value:1
< http://wordpress-develop.dev
> https://wordpress-develop.dev
wp_options.option_value:2
< http://wordpress-develop.dev
> https://wordpress-develop.dev
+------------+--------------+--------------+------+
| Table      | Column       | Replacements | Type |
+------------+--------------+--------------+------+
| wp_options | option_value | 3            | PHP  |
+------------+--------------+--------------+------+
Success: Made 3 replacements.

It’s even more beautiful in color. Check out this asciicinema video for the full glory.

Note: wp search-replace is much slower when logging transformations, so please use it wisely.

See registered image sizes

Does uploading a new image take forever? You might have too many registered image sizes!

Both themes and plugins make use of add_image_size() [ref] to define names for image sizes they expect to use in templating. For each registered image size with a hard crop, WordPress has to create the cropped version on upload. With dozens of image sizes, uploading an image can take tens of seconds.

Use wp media image-size to see all of the image sizes registered to WordPress [#36, #37, #39, #49]:

$ wp media image-size --format=count
55

55 is too many!

Everything else in v1.4.0

New and notable

  • wp cli has-command: Detect whether a command is registered [#4349].
  • wp site (mature|unmature|public|unpublic): Manage mature and public status of a site [#63].
  • wp * (pluck|patch): Fetch and modify serialized data in options and meta [#24, #65].
  • wp user (spam|unspam): Mark a user as spam or not spam [#74, #86, #90].

Command improvements

  • comment list:
    • Improves performance of --format=count [#64].
  • core download:
    • Use --skip-content to download WordPress without default themes/plugins (US locale only) [#37, #40, #41].
  • core update:
    • Makes use of halt_on_error to catch error and release lock [#38].
  • db export:
    • Includes Y-m-d in default export file name [#36].
  • db search:
    • Caters for reserved word column/table names [#40].
    • Changes default delimiter to chr(1) [#46].
    • Fixes match in non-regex case [#45].
    • Lessens context duplication by shortening and appending context if it overlaps with the next match [#55].
    • Avoids displaying default delimiter on regex fail in db search [#56].
  • export:
    • Adds --stdout to write WXR to STDOUT [#13].
    • Adds --max_file_size=-1 to avoid splitting export files [#12, #21].
    • Adds --max_num_posts=<num> to limit number of posts in an export file [#15].
  • import:
    • Avoids use of GLOB_BRACE for compatibility with Alpine Linux [#14].
  • media import:
    • Properly handles query strings on image import to prevent security error [#35].
    • Adds --preserve-filetime argument to support persisting file modification time [#42].
  • media regenerate:
    • Skips non-thumbnailed PDFs & other images rather than fail [#48].
  • package browse:
    • Adds deprecation notice [#36].
  • package install:
    • Supports package names that differ from repository names [#31].
  • package (install|uninstall):
    • Only includes Composer return code when it’s set [#40].
  • package uninstall:
    • Removes repository entry when uninstalling [#37].
  • plugin install:
    • Skips renaming ZIPs coming from a GitHub archive release/tag [#45].
  • plugin list:
    • Wraps all uses of get_plugins() with the all_plugins filter [#31].
    • Includes dropins like object-cache.php when listing installed plugins [#55].
  • plugin search:
    • Only displays pagination message when --format=table [#56].
  • scaffold child-theme:
    • Transforms spaces to underscores when scaffolding a child theme [#40].
  • scaffold plugin:
    • Adds package-lock.json, yarn.lock to distignore template [#57].
  • scaffold plugin-tests:
    • Uses $TMPDIR in bin/install-wp-tests.sh to allow temp directory override [#39].
    • Uses latest branch for test library in bin/install-wp-tests.sh, and always gets the latest major release when 2 digit WP versions are used [#35].
    • Removes XDebug in scaffolded .travis.yml to improve performance [#49].
    • Improves error message when phpunit is run before bin/install-wp-tests.sh [#55].
    • Shows progress when invoking phpcs [#64].
    • Adds PHP 7.0 and 7.1 to scaffolded GitLab configuration [#68].
  • scaffold (post-type|taxonomy):
    • Enhances pluralization by internalizing Doctrine library [#54, #58, #59].
  • search-replace:
    • Adds esc_sql_ident() function to escape column/table names [#23].
    • Adds --regex-delimiter argument and validation for the --regex-flags argument [#28, #29, #30].
    • Adds --report flag so report can be suppressed with --no-report; --report-changed-only flag option to only report changed fields [#32].
    • Avoids displaying default delimiter on regex failure [#40].
  • site delete:
    • Prevent deleting the root site on multisite, which WordPress core doesn’t permit [#73].
  • user import-csv:
    • Permits importing CSV from STDIN [#100].

Framework enhancements

  • Brings codebase inline with WordPress Coding Standards while narrowly avoiding committer mutiny [#4058].
  • Improves AutoloadSplitter regexes [#4422].
  • Add skips argument to WP_CLI\Utils\report_batch_operation_results() [#4429].
  • Provides dictionary-based suggestions for common misspellings [#4392].
  • Introduces new ‘halt_on_error’ to overload exit in WP_CLI\Utils\http_request() [#4383].
  • When extracting, fails back to tar xz when PharData throws an Exception [#4371].
  • Adds support for running commands over vagrant ssh [#4348].
  • Supports upper- and lowercase ‘Y’ when prompting a flag [#4334].
  • Gives suggestions when exiting early on wp help [#4266, #4303].

Contributors to this release (42 total): aaemnnosttv, AaronRutley, ako80218, anhskohbo, atimmer, BhargavBhandari90, danielbachhuber, desrosj, diggy, domantasg, drzraf, eliseferguson, ethanclevenger91, Flimm, fumikito, GaryJones, gitlost, goldenapples, hearvox, Ippey, javorszky, jdub233, johnbillion, kurudrive, Lewiscowles1986, mimosafa, mitchelldmiller, mitraval192, miya0001, n8finch, nameherocom, nextgenthemes, ryotsun, schlessera, shadyvb, Sidsector9, stevegrunwell, szepeviktor, takezou, Umangvaghela, waviaei, wp-make-coffee

WP Plugin & Theme Checksums Project – Update

The WordPress Plugin & Theme Checksums Project has been kicked off!

Here’s a quick run-down of the current state of affairs:

  • The server side of the development will be managed in https://meta.trac.wordpress.org/ticket/3192, whereas the consuming client code development will be managed in https://github.com/wp-cli/ideas/issues/6 for now.
  • Project collaboration and coordination will happen in a dedicated #cli-checksums channel (not created yet, WIP: https://meta.trac.wordpress.org/ticket/3195)
  • It now looks like the most promising approach is to directly build the checksum generation into the existing .org infrastructure, as the code that is generating the ZIP downloads can easily be extended to generate and store the checksums at the same time. @dd32 is currently working on proof-of-concepts to evaluate the best approach for this in https://meta.trac.wordpress.org/ticket/3192.
  • The current timeline for following through with the project is:
    • Planning phase: Oct 3 – Oct 24
    • Implementation phase: Oct 25 – Nov 14
    • Integration phase: Nov 15 – Nov 22
  • Several companies have offered to support this project with development time or other help, and are already invested in differing degrees: DreamHostPagely, Plesk, Savvii.

UPDATE (Oct. 17th): We opted to keep project collaboration and coordination in the general #cli channel for now, instead of in a dedicated channel.

#checksums

Free WP-CLI stickers for your event!

Want to share your love of WP-CLI with your community?

Starting today, you can fill out this form to request stickers (free of charge) to distribute at your WP-CLI-related event (WordCamp, meetup, or otherwise).

Some fine print to be aware of:

  • Requests must be made at least four weeks in advance of the event by an official organizer of the event.
  • Offer is for up to 50 stickers to any geographic region Stickermule can ship to.
  • Stickers must be made available in a public common area and announced at the beginning or end of the WP-CLI session.

Feel free to reach out to danielbachhuber on Slack with any questions. Happy scripting!

Community Summit: Contributing to WP-CLI

“Contributing to WP-CLI” was the second of two discussions we held at the Community Summit. For notes from the first, see details embedded in the feature development post.

We began the conversation by giving an overview to the current contribution process. Notably:

From the introduction, the conversation turned more free-form. In no particular order, some highlights:

  • One big challenge is that WP-CLI is a rather complex project and assumes a lot of knowledge from a contributor. When onboarding new contributors, they have to learn two things: the process for contributing, and how everything works (without reading the code). Although the internals page is reasonably helpful, it doesn’t cover the command execution flow. Having a flow document would be useful.
  • One observation is that potential contributors can enter the project in different ways (e.g. GitHub repo for a custom command vs. a third-party tutorial on how to use WP-CLI). Documentation primarily provides a linear path.
  • We’re seeing some contributors submit documentation but not a ton. It’s unclear whether this indicates the documentation is good enough, or whether the path to contributing documentation is too confusing. It’d be helpful to see more users open questions about the documentation, as a way of validating/improving the content.
  • Another challenge is that generally of WP-CLI tools is that the user knows the abstract problem, but not which command addresses the problem. Related to this:
    • It would be useful if a help command also included the URL for more information.
    • It’d be nice if you could easily see a tree of all commands.

Thanks to everyone who participated!

Good first issues for new contributors

Want to submit your first pull request to WP-CLI? We’ve identified a few good first issues for you to get your feet wet:

Read through the contributing guide for details on how to get started, or join us in the #cli channel with any questions you might have.