I’m attempting to provide before and after screenshots for all (most) visual changes that land in trunk, on multiple devices. These screenshots are used in visual changelogs such as the Today in the Nightly posts. The goals of this effort are to increase awareness of our usability (particularly touch/mobile), make ui review easier, establish a visual archive, and dogfood visual change on multiple devices.
To create the Today in the nightly posts, I work my way through the latest commits looking for visual changes to UI. From the changesets, I visit the tickets. If the tickets don’t have before and after screenshots for both a desktop and a phone (at least), I take screenshots as I test. I upload those screenshots to the ticket. I comment on the ticket with any bugs I find in the process. I often find bugs, especially on phones. If I don’t have time to provide all screenshots, I tag the ticket with needs-screenshots so I can get back to it later. If I see flow problems while testing, I post a visual record to make/flow and crosslink it with the ticket.
If a ticket has at least one screenshot, I tag it has-screenshots. Once all screenshots are collected, needs-screenshots is removed, leaving has-screenshots.
Now that I have tickets with screenshots that are all tagged with has-screenshots, I collect the shots and publish them as “Today in the Nightly” using the tool we’re all making together, WordPress. The process of dogfooding all visual changes, taking screenshots as I go, and turning the screenshots into posts is fantastic recursive dogfooding that reveals bugs, bad flow, and mobile brokenness. Triage, recursive dogfooding, visual archiving, visual awareness, change awareness, flow awareness, and useful posts are products of this virtuous process. My pet names for these acts of recursive dogfooding and visual archiving are kibbling and visual oxygen (VizOx, a riff on the communication is oxygen mantra of p2).
needs-screenshots and has-screenshots are part of a post-commit lifecycle that hasn’t existed before on core.trac. Historically, once a ticket was closed as fixed, it was done. However, diligence remains. has-screenshots and needs-screenshots are applied after a ticket is resolved as fixed and are used as testing signals by flow patrol. We may need other post-commit workflow keywords, but for now I’m seeing how this has/needs pair works out for visual change testing.
If you’d like to help collect visuals, we want before and after screenshots on at least two devices, with one of them preferably being a phone. We’re increasing our mobile/touch awareness by testing and capturing all visual changes on phones. Provide whatever screenshots you can. The Flow Patrol team will handle the remainder. Developers, taking screenshots of your patches and commits as you test really helps ui/ux review and flow patrol. This process is an engine for empathy and awareness.
I’m focused on visual change and usability, but other teams with other focuses might want their own post-commit lifecycle keywords. Commit is not the end of the process.
Obligatory recruiting pitch: If you’re interested in nurturing a QA mindset in a continuous integration, continuous dogfooding environment, help the Flow Patrol team continuously dogfood flow. Some of our duties are listed in the Flow Patrol handbook. Ideally, every component and feature team has someone continuously dogfooding, patrolling flow, publishing visual records, and helping teams meet merge criteria.