5.3 Release Cycle Post-mortem

By the Numbers

This cycle was longer, taking just over six months. The development (alpha) stage was 138 days, from May 8 to September 23. During that time there were 779 commits to 5.3 (trunk). Most active committers were @SergeyBiryukov with 426 (no surprises there), @pento 66, @afercia and @desrosj with 51 commits each.

The “beta” stage started on September 23 and lasted for 23 days. There were 274 commits. Most active committers were @SergeyBiryukov with 78, @afercia 42, @desrosj 34, and @whyisjake with 24 commits.

The “release candidate” stage started on October 16 and lasted for 27 days. There were 100 commits including documentation and build tools changes. Most active were @SergeyBiryukov with 39, @desrosj 20, and @johnbillion with 14 commits. There were 30 commits that were merged to the 5.3 branch after it was created on October 29.

In total the release cycle lasted for 188 days and there were 1,153 commits. Looking at the numbers, the busiest time was during beta with an average of nearly 12 commits per day. For comparison there were only about 5.5 commits per day during the “alpha” stage *.

Needed Improvements

There were also some irregularities during the 5.3 cycle.

  • Many “new” bugfixes were committed during beta. Generally these are added in alpha, and beta is reserved for testing and fixing any bugs introduced in alpha.
  • Several apparent enhancements were committed from partially related bugfix tickets: #34904, #47477, #47153, etc. Preferably there would be a discrete 1:1 relationship between tickets and changesets.
  • Some code that would be considered “alpha” (mostly untested and non-reviewed) was committed during beta. Preferably any code committed in beta is well-reviewed by multiple people with varied knowledge/experience with the existing code/areas of expertise.

A substantial departure from past releases was the inclusion of “global” CSS changes affecting all of the WordPress admin during beta. They were committed from unrelated or partially related tickets: [46241], [46360], [46244], and most of the other changesets marked as “Improve and modernize user interface controls”.

It would have been preferable for the changes to be separated in new tickets, tested, reviewed, and then committed. Separating these into individual tickets would also have let enhancements be isolated and prepared for the next major release.

Individually, none of the irregularities listed above are a big problem, but when taken all together it introduced avoidable complexity. In this particular case, many small styling bugs were introduced during beta, which is far too late in the release cycle. Again, individually they were fixable, but their late introduction and lack of discrete ticketing left folks without enough time to find and fix them all.

Moving Forward

There are few steps that can be taken to avoid such irregularities in the future:

  • Add some “Release Cycle 101” to the developer documentation. This will ensure that people without prior experience are on the same page. Generally during alpha anything goes. Trunk can be broken and fixed every day as the contributors iterate on enhancements and bugfixes. Beta is for testing by the users and to fix the bugs that were introduced in alpha. RC is “we think it’s ready, find the edge cases!” 
  • Add a requirement for a code review for all changes committed during beta. Generally this has been the case for most commits during both alpha and beta. Requiring a review before merging works very well in Gutenberg. It’s about time to make it “official” on Trac too.
  • Remove the “(blessed)” part from the “task” ticket type on Trac. It seems it gives the wrong impression that a ticket can be “blessed” by anybody to allow it to bypass the release cycle rules.
  • Restrict use of the “task” ticket type on Trac only to the project leadership. Bypassing (our own) release cycle rules is not something that can be done lightly and carries a lot of responsibilities. Also, perhaps add a requirement for at least two reviews for each commit/code change in this case.
  • Perhaps extend the section on creating and using Trac tickets to specifically disallow unrelated or partially related changes to be made. This may be somewhat hard to determine, but when in doubt, better to create a new ticket instead of exponentially extending an existing ticket. In 5.3 there are several examples of such “run-away” tickets: #47477, #47153, #34904, etc.


* The slower commit pace during alpha is partially due to new features being developed, and not committed until deemed “ready”. Another reason is perhaps that complex enhancements and bug fixes need more research, reviews and testing before committing. Yet another reason is that perhaps most contributors focus more on the new features, enhancements and more complex bug fixes leaving the “easy” bugs for later.

#5-3, #retrospective