New bootstrapping mechanism to load the WP-CLI framework

The upcoming 1.2.0 release of WP-CLI will include a refactored bootstrapping flow that provides a more flexible way of loading the framework itself and setting everything up. This refactoring is needed to allow for some of the expected functionality to work in conjunction with bundled commands being provided through external packages and the general use of autoloading (Issue #3850 | Pull Request #3872).

What does the term “bootstrapping” mean?

In general terms, bootstrapping code is code that is not part of the actual logic meant to solve the problem, but rather code that is needed to prepare the environment so that the actual logic is able to run.

This usually means locating files and folders, setting constants, reading configuration, etc…

What problem does this change solve?

The current stable release has all bundled commands be part of the wp-cli/wp-cli repository as a big monolithic package. This has several drawbacks, like having a huge testsuite be run for every single change to the framework (and waiting for 2 hours on feedback from the tests), and coupling the update frequency of these commands to the framework itself (making a quick hotfix to one single command difficult).

To get around these problems, we’ve extracted all of the commands into separate repositories that exist on their own, come with their own testsuites and have their own release cycles. To be able to profit from these decoupled release cycles, though, you need to be able to update a bundled command independently of the framework. This means that the act of loading the framework needs to be strictly separate to the act of loading the bundled commands.

The new bootstrapping mechanism separates the procedural list of load operations into a set of isolated steps for which the order can be freely defined.

General flow

After defining some constants to locate the needed files, WP-CLI will load the bootstrap component and call the WP_CLI\bootstrap() function (source).

This function defines a list of bootstrapping steps that will then be loaded and processed in the provided order (source).

The list of bootstrapping steps is a basic array of fully qualified class names (source). Each of the classes implements the WP_CLI\Bootstrap\BootstrapStep interface, meaning that it provides a process( BootstrapState $state ) method (source).

Bootstrap steps

When the process( BootstrapState $state ) method of a WP_CLI\Bootstrap\BootstrapStep object is being called, that object will execute its tasks to take care of its responsibilities and then return the (potentially modified) $state.

Each step works in isolation, so that they can be added/removed/rearranged as needed.

The current code does not allow for the list of steps to be modified from outside code yet, while we collect data and feedback about this new mechanism. However, if we feel confident about the reliability and usefulness, we might open up the array of steps for modification with a later release. If you feel you have a valid use case for such modifications to the bootstrapping process, please let us know!

Bootstrap state

The WP_CLI\Bootstrap\BootstrapState object that is being passed from one step to the next is a very basic key-value store. It currently only knows about one key, BootstrapState::IS_PROTECTED_COMMAND, which is used to skip certain steps when dealing with protected commands (source).

The only protected command right now is wp cli info (source). This protection means that loading of external command packages and --required files is skipped, so that the command can not be overridden. In this way, you can always be sure that the wp cli info command is indeed the one provided by the framework.

Split autoloader

The current implementation uses a custom Composer plugin to split the Composer autoloader provided by the wp-cli/wp-cli package into two separate autoloaders (source). This is needed to allow autoloaders from external command packages to take precedence over the bundled commands. Otherwise, you would not be able to update bundled commands through the built-in package manager.

This is only a workaround to get around the fact that the “WP-CLI Framework” would actually need to be a separate repository from the “bundled WP-CLI”. If the repositories themselves should be split at a later date, this custom Composer plugin will not be needed and removed again. So don’t rely on this plugin or the functionality it provides to be available.

Managing command dependencies

A combination of changes we are currently working on – putting all commands into external packages as well as making the WP-CLI bootstrapping process more flexible – made it obvious that we need to deal with command dependencies in one way or another. As an example, the scaffold package command depends on the scaffold command to be already registered. Both are provided through external packages, though. If they are loaded in the wrong order, adding the scaffold package command will not work as expected.

Therefore, the current nightly includes a new mechanism for dealing with these command dependencies. This is achieved through a pair of new hooks:

Before adding a command – check requirements

before_add_command:<command>

The main purpose of this hook is to allow you to check runtime requirements at the moment the command would be added.

The callback you provide will have a WP_CLI\Dispatcher\CommandAddition object be provided as the first argument. This object has a method abort( $reason ) that you can use to prevent the command from being added. The reason that you provide is not yet being used, but the plan is to display this reason in the help screen so that users have feedback about why certain commands are unavailable.

Usage example:

WP_CLI::add_hook( 'before_add_command:db', function ( $addition ) {
    if ( ! $this->database_is_available() ) {
        $addition->abort(
            'The db command needs a working connection to the database.'
        );
    }
} );

After adding a command – load dependent commands

after_add_command:<command>

This hook is used to have one command depend on a different command being loaded first. The main use case is when you add a new sub-command to a parent command located in a different package. As you cannot simply depend on loading order, you can wrap the addition of this sub-command in the after_add_command:<command>  hook.

Usage example:

WP_CLI::add_hook( 'after_add_command:scaffold', function () {
    WP_CLI::add_command( 'scaffold package', 'WP_CLI\ScaffoldPackageCommand' );
} );

Future considerations

These two hooks allow commands to be built in a more robust way and open up new possibilities for the future.

One of the goals we currently discuss is to have the help screen display unavailable commands together with the reason why the command has been disabled. This provides valuable feedback to the user and might even point to the next action for resolving the issue.

Also, getting rid of some of the run-time errors that let WP-CLI fail without even showing a help screen can potentially be defused through these hooks as well. A simple requirements check before adding a command can then display a warning, and avoid adding the command that would throw an error.

Good first issues for new contributors

Want to submit your first pull request to WP-CLI? We’ve identified a few good first issues for new contributors to get their feet wet:

Read through the contributing guide for details on how to get started, or join us in the #cli channel with any questions you might have. Office hours are today, Tuesday, April 25 at 16:00 UTC

New co-maintainer, Alain — thanks 2017 sponsors

WP-CLI welcomes its second maintainer, thanks to the support of our 2017 sponsors.

New co-maintainer: Alain Schlesser

First, I’d like to formally welcome Alain Schlesser as WP-CLI’s newest co-maintainer. If you’ve been following the project over the last month, you may have noticed him diving deep into WP-CLI’s internals. We’re excited about the infrastructural improvements coming in the next release, and the new features they’ll enable.

Alain is new to the WordPress community, and brings years of experience developing for other platforms, from hobby game development to large enterprise projects in a government environment.

I’m wildly excited about the opportunity to join Daniel and work on WP-CLI. This project represents a special facet of the WordPress ecosystem and comes with a unique set of requirements and challenges. I can’t wait to explore the possibilities!
— Alain

With Alain joining the project as a co-maintainer, the WP-CLI project is restoring capacity to meet current demands (e.g. support), and ramping up on new feature development and evangelization. We’ve already improved the build time by 33%!

Come say hello to Alain at our new office hours every Tuesday. Office hours start tomorrow, Tuesday, April 4 at 16:00 UTC

Thanks 2017 sponsors

Maintaining an open source project requires a substantial, ongoing commitment of time and energy. WP-CLI is a dependable tool loved by thousands because it’s actively maintained, which is made possible by the generosity of its sponsors.

Premier Sponsors

Individual Sponsors

Aaron Jorbin, Anant Shrivastava, Andy Fragen, Austin Ginder, Barry van Someren, Ben Meredith, Ben Welch-Bolen, Bill Christensen, Birgit Pauli-Haack, Bjørn Johansen, Boone Gorges, Carl Hancock, Chuck Reynolds, Craig Martin, Daniel Hüsken, David Herrera, David McDonald, Doruk Fisek, Drew Linsalata, Dwayne McDaniel, Erik Joling, Full Name, Hidetaka Okamoto, Hugh McGuire, Ian Johnson, Jared Atchison, Jason Conroy, Javi, Javier Arnáez, Jeremiah J Terhark, Joel Gaeddert, John Speed, Jonathan Bossenger, Jonathan Daggerhart, Joost de Valk, Joshua Koenig, Kevin Cristiano, Knight Tan, Matt Gross, Matthew Lawrence, Mike Garrett, Miya0001, Ned Zimmerman, Nicholas Duncan, Nick Cernis, Patrick Garman, Per Søderlind, Pippin Williamson, Rachel Baker, Rahul Bansal, Ralf Hortt, required gmbh, Richard Aber, Rick Wiggins, Robert Mathews, Rocket Lift In., Ross Wintle, Ryan Sullivan, Sean Hayes, Simon Blackbourn, Steph Gray, Tadpole Collective, Tanner, Thiana Kitrilakis, Thorsten Ott, Vlad Olaru, WP Bullet, Yash Chandra

RFC: Usage analytics and WP-CLI v2.0.0

There are two issues in the backlog we’d love your input on:

  • Capture and report anonymized usage statistics (scheduled for v1.3.0) – In addition to the data points listed, what others might be worthwhile to capture? And, are there other aspects to the project we should consider?
  • WP-CLI 2.0.0 (scheduled for late 2017) – What breaking changes should we consider and why?

Feel free to weigh in as you see fit.

 

New support venue: WordPress.org support forum

Have a question about something related to WP-CLI? There are now two places you can go for general help:

  • Join the #cli channel in the WordPress.org Slack to chat with whomever might be available at the time. This option is best for quick questions.
  • Post a new thread in the WordPress.org support forum and tag it with ‘WP-CLI‘ so it’s seen by the community.

Want to help others with their use of WP-CLI? Click “Subscribe to this topic tag” to receive email notifications of new forum discussion:

GitHub issues are meant for tracking enhancements to and bugs of existing commands, not general support. Use the wp-cli/ideas repository to up vote existing ideas or share your own.

Planning for Community Summit 2017

In preparation for the 2017 WordPress Community Summit (“CS”) on June 13th-14th before WordCamp Europe in Paris, France we’ve been asked to provide the following (in summary):

  1. A list of topics for the CS
  2. A list of representatives to attend the CS
  3. One or two contributors who are willing to help with the organization of the CS

This post serves as a request for input for the above three areas. Note that these three requests are detailed further with some clarifying notes on Make/Summit.

Feel free to ping me privately in Slack if you prefer. I will send a summary of the responses with the Community Summit team on Wednesday, March 1st (as I’ll be offline on Thursday and Friday).

Two new commands: doctor and profile

Excited to experiment with a couple new WP-CLI commands? wp doctor and wp profile are now available for everyone to install.

wp doctor lets you diagnose problems within WordPress by running a series of checks for symptoms. It includes an API for defining your own diagnosis, as well as writing custom checks.

$ wp @daniel doctor check --all
Running checks  100% [============================================================================================] 0:02 / 0:09
+----------------------------+---------+--------------------------------------------------------------------+
| name                       | status  | message                                                            |
+----------------------------+---------+--------------------------------------------------------------------+
| core-verify-checksums      | success | WordPress verifies against its checksums.                          |
| file-eval                  | success | All 'php' files passed check for 'eval\(.*base64_decode\(.*'.      |
| autoload-options-size      | success | Autoloaded options size (16.25kb) is less than threshold (900kb).  |
| constant-savequeries-falsy | success | Constant 'SAVEQUERIES' is undefined.                               |
| constant-wp-debug-falsy    | success | Constant 'WP_DEBUG' is defined falsy.                              |
| core-update                | success | WordPress is at the latest version.                                |
| cron-count                 | success | Total number of cron jobs is within normal operating expectations. |
| cron-duplicates            | success | All cron job counts are within normal operating expectations.      |
| option-blog-public         | success | Site is public as expected.                                        |
| plugin-active-count        | success | Number of active plugins (2) is less than threshold (80).          |
| plugin-deactivated         | success | Less than 40 percent of plugins are deactivated.                   |
| plugin-update              | success | Plugins are up to date.                                            |
| theme-update               | warning | 1 theme has an update available.                                   |
+----------------------------+---------+--------------------------------------------------------------------+

wp profile quickly identifies what’s slow with WordPress, by giving you visibility into key I/O indicators.

$ wp @daniel profile stage --fields=stage,time,cache_ratio
+------------+---------+-------------+
| stage      | time    | cache_ratio |
+------------+---------+-------------+
| bootstrap  | 0.2643s | 80%         |
| main_query | 0.0186s | 85.71%      |
| template   | 0.0489s | 93.71%      |
+------------+---------+-------------+
| total (3)  | 0.3318s | 86.47%      |
+------------+---------+-------------+

Both packages are in early stages of development — feedback welcome!

Version 1.1.0 released

Happy release day!

Today, I’m excited to bring you WP-CLI v1.1.0, chock full of enhancements and bug fixes.

Want to get props in the next release? There are a few projects we’ll be working on:

And, in case you missed it: I’m looking for help maintaining WP-CLI (paid opportunity, commitment of 5-10 hours/week). Know someone who might fit? Email daniel@handbuilt.co or ping ‘danielbachhuber’ on the WordPress Slack.

Everything in 1.1.0

Command improvements:

  • wp cache *:
    • Explicitly sets default cache group as ‘default’ to replicate WordPress’ behavior [#3714]
  • wp cache type:
    • Detects W3 Total Cache object cache [#3637]
  • wp (comment|post|user) list:
    • Magically parses CSV values for any argument with double underscore [#3726, #3744]
  • wp core config:
    • Introduces --force parameter for overwriting existing wp-config.php file [#3706]
  • wp core is-installed:
    • Prevents error notice from wp_guess_url() when core isn’t installed [#3711]
  • wp core language install:
    • Passes $wp_version through to translations_api() so that WordPress reports the correct language download file [#3748]
  • wp core update:
    • Supports --version=(nightly|trunk), which will download the latest nightly build [#3645]
  • wp core update-db:
    • Updates wpmu_upgrade_site option for all networks, not just current, to ensure the admin notice is dismissed [#3659]
  • wp db *:
    • Runs help at the same point as the command, to enable wp help db import to be run when WordPress has been downloaded, a wp-config.php has been created, but WordPress hasn’t yet been installed [#3780]
  • wp db cli:
    • Makes it possible to pass arguments through to the mysql executable [#3745]
  • wp db export:
    • Appends a random hash to generated SQL file to help mitigate database files from being accessible at predictable URLs [#3765]
  • wp media regenerate:
    • Support generating thumbnails for ‘application/pdf’ for WordPress 4.7+ [#3768]
  • wp plugin install:
    • Correctly installs ZIPs uploaded to the GitHub release page for a project [#3776]
    • Displays error message when a plugin cannot be installed due to write permissions [#3764]
  • wp scaffold plugin:
    • Ignores multisite.xml.dist in scaffolded .distignore [#3677]
  • wp scaffold plugin-tests:
    • Opts-in to container-based infrastructure on Travis [#3739]
    • Supports PHP 7.1 and PHPUnit 5.7 when scaffolding .travis.yml [#3758]
    • Updates GitLab template to run WPCS tests [#3772]
  • wp scaffold (plugin-tests|theme-tests):
    • Validates plugin/theme slug to prevent you from scaffolding tests into an unexpected directory [#3666]
  • wp scaffold _s:
    • Validates theme slug earlier to provide a more understandable error [#3724]
  • wp search-replace:
    • Supports passing regex modifiers [#3639]
    • Only skips replacement when array is a reference to an existing array [#3708, #3713]
  • wp server:
    • Sets the path global parameter as docroot [#3700]
  • wp site create:
    • Replicates core logic for creating $newdomain and $path to form proper path then global $base isn’t ‘/’ [#3688]
  • wp site option list:
    • Adds --site_id=<id> filter [#3769]
  • wp theme install:
    • Uses http cacher when automatically installing a parent theme [#3689]
  • wp theme mod get:
    • Introduces --field=<field> argument for fetching a particular field [#3644]

Framework enhancements:

  • Updates Composer-based dependencies to latest [#3638, #3676#3678, #3698, #3720, #3786]
  • Ensures passed positionals take precedent over defaults defined in wp-cli.yml [#3648]
  • WP_CLI::runcommand() correctly persists current process’ environment variables [#3683, #3730]
  • Removes backslashes from display of path when destination folder already exists [#3691]
  • Globalizes wp-config.php variables to local scope too, enabling WordPress to properly change $table_prefix in multisite [#3695]
  • Adds global parameters to bash completion [#3697]
  • Permits dots to be used in aliases [#3705]
  • Introduce a Behat step for replacing a string in a file [#3712]

Contributors to this release (pull requests, documentation, and package authors): amqbgeihsgt, danielbachhuberedpittol, ernilambargreatislanderinderpreet99, louisremilwhmetodiew, migueldemoura, miya0001mmcev106, nikschavan, ntwbnylenocean90ramoonusrosswintleszepeviktortrepmalwestonruter

You can browse the full list of resolved issues on Github.

New home for WP-CLI docs

WP-CLI’s long-form documentation has a new shiny home: make.wordpress.org/cli/handbook

2017-01-26 at 3.47 PM

Want to suggest an edit?

Click on the “Edit” next to the title to submit your changes via pull request workflow. The documents are tracked in a GitHub repo for easy collaboration. Just like with code changes, documentation contributors receive credit in every WP-CLI release.

Have five minutes to make the documentation better?

Read through the handbook and open an issue with ideas on how we can improve or expand upon our documentation.

Happy scripting!