Release Process Checklists

The release process is complex and beyond one person. Releasing is an intricate dance that we haven’t been sufficiently capturing. Knowledge siloed in heads needs to be committed to public, institutional memory. The upcoming 4.5 release is an opportunity to capture every step of the dance so that we can iterate process, automate away lingering drudgery, and improve our cognitive net for the stressful task of releasing to 25%. I like using checklists in this cognitive net. They relieve anxiety, make process transparent, and help teams flow during stress. We already have a couple release checklists. We can build on those while adopting a little checklist culture in a manner empathetic to developers and flow. Pitch:

Checklist cool tricks

Checklists…

  • distribute power.
  • push power of decision making to the periphery.
  • provide a cognitive net.
  • make the minimum necessary steps explicit.
  • make sure simple steps are not missed.
  • make sure people talk.
  • capture and shape real flow.
  • inspire flow in emergencies and sustain it through the quotidian.
  • capture flow between teams.
  • encourage a shared culture around flow.
  • accessibly capture institutional memory in the context of flow.

Attributes of a good checklist

What makes a good checklist? Checklist shouldn’t be about just checking boxes. Instead of being a chore and an admonishing finger, checklists should fit and assist real flow. The Checklist Manifesto offers these suggestions. Ideally, checklists…

  • are not lengthy.
  • have clear, concise objectives.
  • define a clear pause point at which the checklist is supposed to be used.
  • have fewer than ten items per pause point.
  • fit the flow of the work.
  • continually update as living documents.

See this checklist for checklists and this example checklist for more.

Stuff to checklist

The major release checklist attempts to use pause points and follow the suggestions above. The major and minor release checklists are pretty rough and incomplete and overlap with each other. These and the things to keep in mind list need love and unification with help from developers who are in the release flow and handling controls on the release train.

about.php is…quite the process. It needs the oxygenating powers of a checklist.

Checklist Feature plugin merges.

Checklist bundled theme releases so stuff like this makes it into institutional memory.

Beta and RC releases.

Plenty of other stuff. 🙂

Start by capturing. As we walk 4.5 release flows, capture.

Selected quotes from The Checklist Manifesto

Checklists supply a set of checks to ensure the stupid but critical stuff is not overlooked, and they supply another set of checks to ensure people talk and coordinate and accept responsibility while nonetheless being left the power to manage the nuances and unpredictabilities the best they know how.

Continue reading

#checklists, #pitch, #process, #release-process

#needs-screenshots, #has-screenshots, and multi-device dogfooding of visual change

I’m attempting to provide before and after screenshots for all (most) visual changes that land in trunk, on multiple devices. These screenshots are used in visual changelogs such as the Today in the Nightly posts. The goals of this effort are to increase awareness of our usability (particularly touch/mobile), make ui review easier, establish a visual archive, and dogfood visual change on multiple devices.

To create the Today in the nightly posts, I work my way through the latest commits looking for visual changes to UI. From the changesets, I visit the tickets. If the tickets don’t have before and after screenshots for both a desktop and a phone (at least), I take screenshots as I test. I upload those screenshots to the ticket. I comment on the ticket with any bugs I find in the process. I often find bugs, especially on phones. If I don’t have time to provide all screenshots, I tag the ticket with needs-screenshots so I can get back to it later. If I see flow problems while testing, I post a visual record to make/flow and crosslink it with the ticket.

If a ticket has at least one screenshot, I tag it has-screenshots. Once all screenshots are collected, needs-screenshots is removed, leaving has-screenshots.

Now that I have tickets with screenshots that are all tagged with has-screenshots, I collect the shots and publish them as “Today in the Nightly” using the tool we’re all making together, WordPress. The process of dogfooding all visual changes, taking screenshots as I go, and turning the screenshots into posts is fantastic recursive dogfooding that reveals bugs, bad flow, and mobile brokenness. Triage, recursive dogfooding, visual archiving, visual awareness, change awareness, flow awareness, and useful posts are products of this virtuous process. My pet names for these acts of recursive dogfooding and visual archiving are kibbling and visual oxygen (VizOx, a riff on the communication is oxygen mantra of p2).

needs-screenshots and has-screenshots are part of a post-commit lifecycle that hasn’t existed before on core.trac. Historically, once a ticket was closed as fixed, it was done. However, diligence remains. has-screenshots and needs-screenshots are applied after a ticket is resolved as fixed and are used as testing signals by flow patrol. We may need other post-commit workflow keywords, but for now I’m seeing how this has/needs pair works out for visual change testing.

If you’d like to help collect visuals, we want before and after screenshots on at least two devices, with one of them preferably being a phone. We’re increasing our mobile/touch awareness by testing and capturing all visual changes on phones. Provide whatever screenshots you can. The Flow Patrol team will handle the remainder. Developers, taking screenshots of your patches and commits as you test really helps ui/ux review and flow patrol. This process is an engine for empathy and awareness.

I’m focused on visual change and usability, but other teams with other focuses might want their own post-commit lifecycle keywords. Commit is not the end of the process.

Obligatory recruiting pitch: If you’re interested in nurturing a QA mindset in a continuous integration, continuous dogfooding environment, help the Flow Patrol team continuously dogfood flow. Some of our duties are listed in the Flow Patrol handbook. Ideally, every component and feature team has someone continuously dogfooding, patrolling flow, publishing visual records, and helping teams meet merge criteria.

#awareness, #dogfooding, #empathy, #flow, #has-screenshots, #kibbling, #needs-screenshots, #post-commit-flow, #vizox, #workflow

Component Page Updates for 4.4

Now that 4.4 is underway, let’s update the component pages to reflect 4.4 activity. The Customize, Editor, and Press This pages serve as good templates, though they all need 4.4 updates. The component pages are targeted at beta testers. They should describe the component, list milestones (roadmap), and explain what needs testing and how to test it. Good component pages assist triage. For details, see the previous round of component page updates.

Also, if your component has a corresponding Slack chat, link to the component page from the chat’s channel topic. This assists using Slack in beta testing flows.

Component maintainers, here are your component pages…

Continue reading

#components, #maintainership

Today in the Nightly: Site icons in the customizer, editor patterns, more accessible comment bubbles, row toggle focus styling

Install the nightly, and try out this fresh batch of shiny.

Site Icons in the Customizer

I’ve long wanted site icons in the customizer alongside site title and tagline. The identity information that I always want to edit when first setting up a site are now all together in the customizer.

For more visuals, see these visual records.

See #16434.

Editor Patterns

Create bulleted lists, ordered lists, and blockquotes using markdown like patterns. I find this particularly handy on phones when the editor toolbar is offscreen.

Screen Shot 2015-07-14 at 4.39.12 PM

See #31441.

Better focus styling for list table row toggles

See #32395.

Better accessibility and design for the comments bubble

The comments columns in our list tables were among the most confusing for screen reader users. Accessibility and visuals are now improved.

See #32152.

Eliminate content overruns on small screens

An audit of content overruns on small screens resulted in many fixes.

After:

Before:

See #32846.

Styling improvements on small screens for Right Now in the network admin

See #32962.

Improved header information in Network Admin Edit Site tabs

  • Use the site’s name rather than URL in the Edit Site header.
  • Provide “Visit” and “Dashboard” links for the site on all tabs.

After:

Before:

See #32525.

Disambiguate “Automatically add new top-level pages to this menu”

In the customizer, a menu’s auto-add pages option is now separated from the preceding menu location checkboxes.

See #32820.

 Passwords UI Improvements

Passwords received a couple of improvements. The show/hide toggles look better, and passwords ui is on the install screen. Passwords on the install screen still needs a little more flow work.

See #32589 and #32925.

For more visuals, see these visual records.

Reduce link noise in media library list view

This is visually subtle but removes confusion for screen readers.

KL_7dmW58c

See #32254.

 

Previously: Today in the Nightly: Customize in the Toolbar, Passwords UI, List Tables on Phones, Dashicons

#accessibility, #bubbles, #comments, #content-overrun, #customize, #edit-site, #editor, #list-tables, #media, #menus, #multisite, #network-admin, #right-now, #site-icons, #today-in-the-nightly

Today in the Nightly: Customize in the Toolbar, Passwords UI, List Tables on Phones, Dashicons

Development leading up to the first beta brought several visual changes. These are available right now in the nightly build. Switch a site to nightly builds and try them out.

Customize in the toolbar

To disambiguate between links to the Customizer and links to the Appearance screens from the front-end, Customize now has a top-level toolbar button rather than having links to it mixed with dashboard links in the site menu. This context mixing leads to disrupted user expectations as they navigate, as well as experiences that feel slower or actually are slower in some cases. See #32924.

This means an additional top-level menu item, but the existing links to Widgets and Themes in the dropdown will now point to the admin, as the Dashboard and Menus links do. The advantage and goal for this change is to make it clear that you are about to enter the customizer. Deep links have not been added back in this go-round; this means that direct links to Header and Background are currently absent (with a very narrow exception related to old browsers). Those two deep links are still available in the admin menu under Appearance, which similarly mixes context but has not yet been addressed.

More changes are coming to the toolbar. Peek at a possibility for more general improvements to the toolbar, being discussed in #32678.

Phone friendly list tables

List tables now scale down to phones. The column truncation strategy they used before didn’t scale down to small screens. A single column layout with disclosures is the new strategy. Some of our most important screens use list tables, notably Media and Posts. Truncated columns was number 5 on our top  5 impediments to flow on touch devices list.

After:

Before:

See #32395.

For more screenshots, see these visual surveys of the list table screens.

Toolbar interaction fixes for touch devices

I’ve been wanting this one for a long time.  Toolbar interaction was number 3 on the top  5 impediments to flow on touch devices.

 

Fixed!

Fixed!

See #29906. That ticket is a good read.  It has: Visual feedback and visual surveys. Punting a working fix to the next release so that a new, more future proof approach could be tried. Development of touch capability detection. Working around iOS. Development of testing checklists. Lots of iteration.

Passwords UI

The password set/change UI was updated with these improvements.

  • Generate the password for the user
  • More tightly integrate password strength meter
  • Warn on weak passwords

See #32589 for more screenshots.

Dashicons update

Dashicons received a big update.

New icons:

  • .dashicons-admin-customizer (f540)
  • .dashicons-admin-multisite (f541)
  • .dashicons-editor-table (f535)
  • .dashicons-filter (f536)
  • .dashicons-hidden (f530)
  • .dashicons-image-filter (f533)
  • .dashicons-image-rotate (f531)
  • .dashicons-layout (f538)
  • .dashicons-sticky (f537)
  • .dashicons-thumbs-down (f542)
  • .dashicons-thumbs-up (f529)
  • .dashicons-unlock (f528)
  • .dashicons-warning (f534)

Updated icons:

  • .dashicons-plus (f132)
  • .dashicons-yes (f147)

dashicons-preview

See #30902.

Better styling for .form-invalid inputs

See #32490.

Responsive styling for my-sites.php

My Sites now moves to a single column layout on narrow viewports. Here it is on an iPhone 6, an iPad, and a Macbook, as well as at full-width.

Here’s what it looked liked before.

my-sites-before

See #31685 – Better responsive styling for my-sites.php

Crosslinking Customizer Panels

The graf in the Menus panel details about using the Customize Menu widget now links directly to the widgets panel.

customize menus details

See #32742.

You might notice the misaligned question mark icon on that screenshot. #32733 is tracking that.

 Easy switching between production and nightly builds

The beta tester plugin makes it easy to switch a site to nightly builds. Now switching back to the latest stable build is just as easy. It’s not the prettiest, but it is shown only to beta testers and will suffice until we finally refresh the Grand Unified Updater screen. For info on using the beta tester plugin to test with nightly builds, see the Beta Testing page of the core handbook.

See #32613.

 

Previously: Today in the Nightly: Site Icons, Text Editor in Press This

#beta-testing-flow, #customize, #dashicons, #list-tables, #multisite, #passwords, #today-in-the-nightly, #toolbar

Today in the Nightly: Site Icons, Text editor in Press This

Here are a few cool things that recently landed in trunk. They are available right now in the nightly build. Install the nightly, and try them out.

Site Icons

We’ve wanted site icons in core for a long time. #16434 was opened four years ago and will be resolved as fixed for 4.3.

 

Our crop controls are not easy to use on my iPhone 6+. The images overflow the right side of the screen. Horizontally and vertically scrolling an image bigger than the screen while working a rubber band select that resets when the image is tapped is not pleasant.

Provide feedback on #16434 or on this post.

See these visual records for more screenshots and flow storyboards.

Text editor in Press This

Press This now has a Text editor for editing HTML, just like the standard editor in post-new.php.

32706.1-wide

Provide feedback on #32706, in #core-pressthis, or on this post.

 Padding for image settings

The Image Crop and Thumbnail Settings boxes received a little bottom padding.

And so that we are always aware of what our mobile experience looks like, here are those settings boxes on an iPhone 6+.

When you see a sidebar obscuring content on a phone, you can be pretty sure you’re witnessing lingering desktop bias. These screens were designed for desktops where you have room to use  sidebars. You can’t make a screen responsive and call it ready for a phone. The image flow effort is working on this.

Provide feedback on #31845 or on this post.

Manage in the Customizer

Appearance > Menus received a “Manage in the Customizer” button to match Appearance > Widgets.

Screen Shot 2015-06-30 at 3.16.57 PM

Next, fix up mobile.

image

Provide feedback on #32808 or on this post.

 

Previously: Today in the Nightly: Inline link toolbar and Press This split button

#4-3, #customize, #editor, #image-editor, #menus, #press-this, #site-icons, #today-in-the-nightly

Maintaining Core Component Pages

https://make.wordpress.org/core/components/

That page lists our components and their maintainers. I use this page when determining which component to use when creating tickets and when picking triage tags for make/flow posts. The component pages linked from there offer useful information to contributors. I’d like to make them more useful. The customizer component has set a good example with their page.

https://make.wordpress.org/core/components/customize/

Screen Shot 2015-06-14 at 10.50.47 PM

If you’re a component maintainer, give your component page some attention. Make it useful to prospective contributors looking to get involved and follow along. If you use github as part of your development flow, link to your github resources. Also, if your component has a slack channel, link to your component page in your channel topic. If you need make/core editing privileges, let us know.

Trust, Live Preview, and Menus in the Customizer

One of the most important things in WordPress is users being able to feel confident as they make changes to their content and more generally to their sites. Being able to make non-destructive changes and preview them is an important component of building that trust. This is perhaps most noticeable in the “save and surprise” approach of the widgets admin screen – every time you add a new widget, modify its settings, or move one around, the changes are saved and appear live on your site, even if you’re not ready yet. The customizer is our framework for live previewing changes. We are committed to providing live preview for all aspects of site customization and making it usable on all devices from phones to large screens.

The customizer is the result of a tremendous amount of work over many releases. It was first introduced in 3.4. In 3.9, it received its first big updates in the form of widgets support and improved header upload and cropping. 4.0 brought panels and contextual controls. Development really started to take off in 4.1 when JS-templated customizer controls and a JS api were introduced, making possible an ecosystem of live preview compatible plugins and themes. 4.2 followed that up with two important features, theme switching and mobile support.

That brings us to today and the ongoing 4.3 development effort. Revamped navigation for the customizer is already in trunk and the nighty builds. The menu customizer feature plugin is a merge candidate for 4.3 and could land soon. These marquee features further our commitment to live preview and need all of the attention we can muster.

The customizer has come a long way, but it still lacks some features and needs time to mature. We have many improvements planned and in-progress, including transactions, partial refresh, theme installation, speedier loading, scaling to large screens, and possibly even integration with front end editing. Our live preview framework offers many possibilities.

Meanwhile, the Appearance screens will remain and will be maintained. Appearance > Menus recently received some attention in the form of a few fixes. More attention is needed and will be given. There are still differences in the flows each approach best enables, whether it’s new site/theme setup, small maintenance tasks, or dedicated content managers for heavy usage of widgets, menus, or other pieces of content that benefit from having a preview mechanism. We should gather quantifiable metrics when it comes to performance and time to completion for a given flow, as well as evaluating the less-objectively-quantifiable perceived performance. There may come a time where the worlds converge; however, that time is not now.

The feature plugin merge window is currently scheduled for June 17th. We have 8 days to get the Menu Customizer plugin ready for merge. Feature plugins must meet several criteria to be considered for inclusion in a release. To meet this criteria, the flow team has started testing and documenting flow and visuals for the menu customizer as well as the recently landed navigation changes. Merge criteria include identifying flows through the customizer, creating visual records of those flows, and creating flow comparisons of existing flow through Appearance > Menus versus flow through the customizer. This is a great and necessary way to contribute that requires no coding. All you need to do is take screenshots and publish them as a captioned gallery using the tool we’re making together, WordPress. We endeavor to be an Alan Lomax of flow, capturing and cataloging real user scenarios. Please help us capture the flows through Appearance > Menus used by you and your clients. We need this information to ensure our new interfaces are mindful and aware of how WordPress is actually used. Information on how to test and contribute visual records is available on the 4.3 development tracking page.

@ryan, @helen, @designsimply

 

#customize, #live-preview, #menus, #preview

We must be our own beta audience.

Using the beta tester plugin, I follow trunk through every phase of development via five devices (iPhone 5, iPhone 6+, Nexus 5, iPad Air, Macbook). With the plugin installed, select “Bleeding edge nightlies”. Every day, your site will auto update to the latest nightly build. We committed long ago to ensuring that trunk is continuously dogfoodable and quickly fixed in the rare event that something nasty happens, like a fatal error. I have never once experienced loss of content while following trunk.  If you follow this blog, consider putting the beta tester plugin on a real site that you use regularly. We must take this small extra risk with our own sites so that we truly see what we’re making before releasing it.

We desperately need mobile beta testers. WordPress will be a champion of the open mobile web. We will work around the iOS Safari bugs that hamper us. We will make the mobile web a better place. With the beta tester plugin installed on a public site, testing betas from any touch device is as straightforward as testing from the desktop. Despite this ease, 4.2 in its current state is a regression from 4.1 on mobile, particularly on iOS. We’ll work through these problems before release, but we really need mobile beta testers as well as mobile patch testers. Mobile patch testing (and patch testing in general) is not so easy to set up. We need better tools and documentation here.

Check out the Beta Testing section of the Core Development Handbook for help setting up the Beta Tester plugin. I’m always hanging out in the #core-flow Slack channel as @boren if you have questions. Let’s build a mobile beta audience.

#beta, #beta-testing-flow, #testing

Mobile patch testing with VVV and xip.io

I recently started using Varying Vagrant Vagrants and xip.io for real device mobile testing of patches on trac tickets. I go through tickets with patches that change UI and drop mobile and desktop screenshots of the patches in action. These screenshots hasten UI feedback and usually reveal visual glitches on mobile that are then patched up, making our mobile experience that little bit better. Until that blue sky someday when I can apply patches to a patch server with a tap, I’ve found VVV and xip.io to be the easiest way to do localhost testing of patches from mobile devices. I’m using Vagrant’s public_network option along with xip.io. This allows me to test from any device on my local network using links like http://build.wordpress-develop.192.168.1.11.xip.io/ without the need for port forwarding, dynamic dns, or hosts file editing. The two resources I used most in setting this up were:

https://github.com/Varying-Vagrant-Vagrants/VVV#the-first-vagrant-up

https://jeremyfelt.com/2015/01/31/various-networking-configurations-vvv/

Working those into something for the core developer handbook would be a great public service. Patch testing and mobile testing need to be much easier. Let’s all start putting mobile screenshots of our patches on trac and dogfood the mobile patch testing experience. Setting up and using npm and grunt are currently undocumented in the handbook. There is much confusion about build vs src. Getting to where I could localhost test patches was a struggle.

How do you localhost test patches from physical mobile devices? Do you also use VVV? Have a cool bash_aliases or other script to share?

 

#beta-testing-flow