Francesca Marano 10:33 am on October 29, 2020
Tags: release-process ( 24 )

Discussion: align the WordPress release cycle with the industry standard

The standard software release life cycle hasn’t really changed much since the beginning of programming.

WordPress, for the most part, attempts to closely follow the established pattern, with the exception of what can get committed in BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process..

For the past year, there have been conversations in Slack and in the blog about the release cycle. Here is a recap post with a call for feedback.

The WordPress Release Cycle – Today

Each release cycle spans multiple weeks and different phases.

Phase 1: Planning and securing team leads

This period starts as soon as a branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". is created for the previous version (the code is copied from “trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision.” to a new “branch” in the repository), which means that two major WordPress versions are worked on at the same time. For example, active development on WordPress 5.6 started about two weeks prior to the WordPress 5.5 release.

During this phase, the release leadRelease Lead The community member ultimately responsible for the Release. discusses features for the next release of WordPress. Contributors get involved with that discussion. The release lead will identify focus leads for each of the features. Information is gathered from different sources to put together a scope and schedule, followed by a kickoff post.

Phase 2: Development work begins

The kick-off post (see an example from WordPress 5.6) marks the beginning of structured, organised work towards Beta. The length of the period might vary, based on what was achieved during the period before the announcement and what is planned for the release.

Focus leads assemble teams and work on their assigned features. Regular chats are scheduled to ensure the development keeps moving forward.

Phase 3: Beta

Betas are released and beta-testers are asked to start reporting bugs. No more commits for new enhancements or feature requests are allowed for the rest of the release.

WordPress generally plans for multiple betas.

Phase 4: Release Candidaterelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta).

In WordPress, release candidates are considered code complete, so no new source code is introduced unless it is to fix regressions or serious bugs discovered during RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta).. Even then, every commit during this phase needs sign-off from two CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. committers.

This means hard-freeze on all strings, including those in the About page. It is important to do so to allow the Polyglots team to have enough time to translate it.

During this phase, a new branch will be created thus starting a new release cycle while we are finishing the current.

Phase 5: Launch

This is the final version that is launched and available for update through the dashboard or via download.

How to align the WordPress release cycle with the industry standard

With this in mind, here are the changes that could be put in place to align the WordPress release cycle with the traditional software release cycle.

Phase 1: Preliminary Planning

Stays the same in terms of tasks. Can be renamed “Preliminary planning” for brevity and clarity.

Phase 2: Alpha

While traditionally priority has been given to enhancements and new features, it would be beneficial to address also all the bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority.-fixes that are planned for inclusion in the version of WordPress people are working on.

Changes

Rename to “Alpha” for brevity and clarity.
All the bugs that people want to see fixed in the next release need to be addressed in this phase and not postponed to Beta.

Phase 3: Beta

This is where WordPress differs from the standard.

Historically,

Beta phase generally begins when the software is feature complete but likely to contain a number of known or unknown bugs.
Wikipedia

Beta is essentially for testing and fixing bugs discovered after the release of Beta 1 (so bugs introduced in Alpha).

Changes

Hard freeze on Beta (ETA code and strings), except for the About page.
The tickets that were not committed and are still on the milestone at the time of Beta 1 release party, should be moved to a later release, even if they are bugfixes.

Benefits

Virtually no new bugs are introduced during Beta. This means focus on testing and subsequent bug fixing.
More efforts and resources could be allocated to beta-testing.
Beta and RC could be and should be fewer and shorter. This is because we address only bug fixes that emerged during beta testing.
Aligning to the norm makes it easier for new people to join since they are used to a certain procedure and they won’t be confused by the way releases are done in WordPress.

Phase 4: Release Candidate

No changes

Phase 5: General Release

Stays the same in terms of tasks. Can be renamed to “General Release” for clarity.

It’s your turn!

Disclaimer: even if everyone is on board, nothing will change for the current release cycle, WordPress 5.6, because we are already in Beta.

Please post feedback on:

Name changes for Phase 1, 2 and 5
All bug-fixes milestoned to be addressed in Alpha and not postponed to Beta
Reserving Beta for testing and fixing bugs discovered by testing

Deadline: ~~November 17th, the date scheduled for Release Candidate 1 of WordPress 5.6 and, potentially, when trunk gets branched for 5.7~~. Extended to December 2nd, to give people more time to respond.

Thanks!

Props @azaozz, @davidbaumwald, @joostdevalk and @ireneyoast for providing historical and technical knowledge on release cycle procedures, and @chanthaboune and @jeffpaul for editing suggestions.

#release-process

Andrew Ozz 11:09 am on October 29, 2020

Thanks @francina for putting this so nicely together. Big +1 for all changes 🙂

Reserving BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. for testing and fixing bugs discovered by testing

This also respects the efforts of the beta-testers by not introducing new bugs in areas they’ve already tested.
Riad Benguella 12:23 pm on October 29, 2020

Does this change come with changes to timelines and workflows?

– The new proposed betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. cycle might not require as much as time as today.
– The time for working on planned tickets is smaller with this proposal if we stick to the current workflows, so how about changes in branching policies: Potentially we could say, working on enhancements and features never stops on trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. but the work on the releases happen on dedicated branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". starting from beta?
- marybaum 5:55 pm on October 29, 2020
  
  This makes total sense. Also, all the hoopla around kicking off a new cycle transfers to the beginning of Alpha, and BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. minus ten is just Beta minus ten.
  
  I think that would also make it easier for potential contributors who might get involved in support of a feature to join the crew for that feature — and then stay because we’re such a fun group to hang out with. 😜
  
  Diversity in software and design interest can also be a thing, said the old lady in the tennis shoes.
- Jake Spurlock 9:03 pm on November 2, 2020
  
  – The time for working on planned tickets is smaller with this proposal if we stick to the current workflows, so how about changes in branching policies: Potentially we could say, working on enhancements and features never stops on trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. but the work on the releases happen on dedicated branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". starting from betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process.?
  
  A big +1 from me here. I think we need to create the release branches at beta to allow for more time and testing against the release code.
Timothy Jacobs 2:21 pm on October 29, 2020

What is the problem we are trying to solve here?
Virtually no new bugs are introduced during BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process.. This means focus on testing and subsequent bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. fixing.

Do we have data indicating that this is a major issue? I’d like to see, for instance, the amount of `defects` that were fixed during beta period, and reopened after say one day.

My concern is that without a dramatic shortening of the release cycle, cutting it in-half. And a commitment to a “tik-tok” release cadence where we focus on a bug fix / stability release after every major feature release, this will simply leave us with less bugs being fixed.

With 3,800 bugs on tracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress., I think changing release cycles in a way that makes it less likely for bugs to be squashed is a risky endeavor.
- John Blackbourn 3:03 pm on October 29, 2020
  
  I don’t think looking at historical data for this is particularly relevant. A bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. can be introduced during betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. and as a result not detected until after the stable release. In order to get an accurate indication of this pattern we’d need to assess every bug report and look at what point during any given release the bug was introduced.
  
  The wider software industry has done this work for us though. A mature software project is one that has a beta period during which the focus is on testing changes made during this release cycle to ensure its stability. Our current workflow means a random bugfix can go in ten minutes before RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta). 1.
  
  Regarding shortening the release cycle, this is addressed:
  
  Beta and RC could be and should be fewer and shorter. This is because we address only bug fixes that emerged during beta testing.
  
  As a result, whether we subsequently increase the frequency of releases or increase the length of alpha is a matter for the project leads.
  - Andrew Ozz 4:17 pm on October 29, 2020
    
    The wider software industry has done this work for us though. A mature software project is one that has a betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. period during which the focus is on testing changes made during this release cycle to ensure its stability.
    
    This! 🙂
  - Timothy Jacobs 4:42 pm on October 29, 2020
    
    If this isn’t a data driven decision, then what is it driven by? And again, what is the problem statement?
    The wider software industry has done this work for us though.
    
    The wider software industry is notorious for letting feature development take priority over fixing a backlog of bugs. What our BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. system essentially allows is for a period of work that is dedicated to bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. fixing and in my opinion is a great benefit to us as a project.
    Regarding shortening the release cycle, this is addressed:
    
    What does shortening the release cycle by two weeks practically mean? Where is that two weeks going? At the beginning? That just means time for more feature development.
    As a result, whether we subsequently increase the frequency of releases or increase the length of alpha is a matter for the project leads.
    
    Without implementing a change to the release cycle length and scope at the same time as restricting bug work, I don’t understand the purpose of making this change now.
    
    TLDR: IMO our definition of beta allows for contributors to have a time where feature development ( which has a deadline! ) is frozen and grants the ability to focus on bugs.
    
    If we don’t want to make any changes in beta, then we should freeze feature development 2+ weeks before beta to grant contributors this time.
    - Andrew Ozz 8:43 am on October 30, 2020
      
      If we don’t want to make any changes in betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process., then we should freeze feature development 2+ weeks before beta to grant contributors this time.
      
      Right. Thinking it’s a good idea to have a “Feature freeze” cutout time, just lets not call it “Beta” 🙂 Two-three weeks before beta-1 sounds like a reasonable timing and would match the current workflow.
      
      The difference between “Feature freeze” and “Beta” would be that WP won’t ask the beta testers to test code that is not ready for testing.
      - Timothy Jacobs 5:12 pm on October 30, 2020
        
        👍 If that change is made, then IMO we should consider branching at BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. 1.
    - Andrew Ozz 8:57 am on October 30, 2020
      
      And again, what is the problem statement?
      
      The main problem (imho) is that the WP release cycle undermines the efforts of the betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. testers by asking them to test code that is not ready for testing.
      
      Currently in WP after beta-1 comes out, the code is still being actively changed/patched but without a focus on testing changes made during the current release cycle and ensuring its stability. In that terms in WP “Beta” is still an extended “Alpha” with some restrictions.
      
      The problem here is not with the workflow. It is mostly with the naming/organizing of the development stages and clearly communicating when the new release is ready for testing.
  - Timothy Jacobs 5:59 pm on October 29, 2020
    
    Also looking at some other projects, our process doesn’t seem that odd for OSS.
    
    Drupal allows for “non-disruptive” bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. fixes in BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. period: https://www.drupal.org/about/core/policies/core-change-policies/allowed-changes-during-the-drupal-8-and-9-release-cycles#beta
    
    AFAICT, PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher does as well. It doesn’t branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". until RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta).: https://github.com/php/php-src/compare/PHP-8.0…master, https://wiki.php.net/todo/php80
- Andrew Ozz 4:23 pm on October 29, 2020
  
  Do we have data indicating…
  
  Statistical data – no, but have a look at the CSSCSS Cascading Style Sheets. changes committed during 5.3-betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process.. Some of the bugs were caught and fixed later during beta and RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta)., others in 5.4, and few “smaller annoyances” still remain.
  - Timothy Jacobs 4:32 pm on October 29, 2020
    
    How does that compare to the number of bugs that usually get fixed in the betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. period without issue? Looking at 5.5, more than 134 bugs were fixed during the beta period.
    
    I think the CSSCSS Cascading Style Sheets. issue shows that we need to be better about determining which bugs are too large to land in beta. Not a blanket ban on beta bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. fixing.
    - Andrew Ozz 2:00 pm on October 30, 2020
      
      Tried to get some stats about the 5.5 dev. cycle. Pulled a list of all tickets that were closed as fixed during 5.5-beta and RC. Didn’t spend much time on it, just removed duplicates (from committing to the new branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch".) and marked some of the non-code changes. Tickets that were worked on during 5.5-alpha still need to be marked/counted.
      
      The stats are:
      
      Tickets opened before the 5.5 dev. cycle and fixed during 5.5-betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. and RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta).: Total 49, 29 code changes and 20 non-code or previously committed.
      
      Tickets opened during 5.5-alpha and fixed during beta and RC: Total 32, 27 code changes and 5 non-code or previously committed.
      
      Tickets opened and fixed during beta: Total 70, 37 code changes and 33 non-code.
      
      Tickets opened and fixed during RC: Total 23, 15 code changes and 8 non-code.
      - Timothy Jacobs 5:14 pm on October 30, 2020
        
        That’s good data to have. I think it shows the value of having a feature freeze period where non-trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. fixing is allowed.
justlevine 5:17 pm on October 29, 2020

The name change is a no-brainer for me. Will make it a lot easier for new developers to acclimate.
I like the idea of an Alpha phase ending features, but as others echoed: without data showing there’s an issue with bugfixes being pushed during the betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process., I think all those bugs in tracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. take priority.

As far as shortening both the beta and rc, while that would be great in theory, I don’t think it’s practical irl. There is already a struggle with getting all the release docs witten before RC1, not sure how the docs team would manage with a shorter beta period.
bridgetwillard 5:29 pm on October 29, 2020

I think it’s fantastic.

I’d also like to suggest no updates in a calendar after 15 Oct.

PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and theme developers as well as agencies don’t much of a holiday if it drops on Thanksgiving like PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher 8 is this year or just after like 5.6 is (8 Dec).
loru88 7:48 pm on October 29, 2020

I like the plan, I Just have a question: why “General Release” instead of the most common “General Availability” (GA)?
Garrett Hyder 6:21 am on October 30, 2020

Personally I feel it beneficial to have a separate deadline for enhancements/features and bugs as this allows developers to switch focus after the first deadline to then address a slew of outstanding bugs.

I wonder if we can keep more of the current model but introduce an alpha and alpha 2 for early and standard enhancements respectively. Followed by the betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. phase where general bugs are fixed in beta 1&2, and beta 3&4 are focused on trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. bugs or critical items. Beta 1&2 could be seen as alpha 3&4 if we want to hold off on the call for beta testers until all ‘new’ items (enhancements/features/pre-existing-bugs) should be introduced.

With a single deadline on both enhancementenhancement Enhancements are simple improvements to WordPress, such as the addition of a hook, a new feature, or an improvement to an existing feature./features and bugs I fear the focus of maintainers and committers will be on the larger items. This disadvantages the new contributors and their chances of their first prop as the majority of them are starting small with the pre-existing bugs.
- Andrew Ozz 9:32 am on October 30, 2020
  
  I feel it beneficial to have a separate deadline for enhancements/features and bugs
  
  Yes, this is a good idea (also see above comment). Having a “Feature freeze” is an accepted practice in software development.
  
  This is also how releases work in WP currently (more or less). The only thing missing is “formalizing” the names/stages.
Erin 'Folletto' Casali 12:53 pm on November 19, 2020

Yes.
It’s clearer, communicates better, and provides more structure while reducing complexity.
Sergey Biryukov 3:00 pm on November 19, 2020

Thanks for this proposal!

Similar to what some others have noted above, my concern here is to not shorten the time allowed for fixing older bugs, which I see as an essential part of the project.

As long as we branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". at BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. 1 to allow for continuous development, where the branch is reserved for regressions and trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. is open for any commits to keep the momentum on tickets that were punted, I support these changes.
Carolina Nymark 5:23 am on November 25, 2020

I welcome a structure where the freeze is more clear and not arbitrary.

Something I had internal struggles (and sometimes not so internal…)
with both 5.5 and 5.6 releases is that the people I respect, listen to and go to for advice and help have different opinions about when a piece of code or text can be changed.
Person A: You need to change that
Person B: You can’t change that

Changes being made during the betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. cycle and RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta). are a big problem,
Up until RC1 the theme squad has played catch up with the changes made to the editor and it has been difficult, there has been no stability.
Ari Stathopoulos 6:00 am on November 25, 2020

Thank you for this suggestion Francesca!

It always seemed weird to be that the terms alpha/betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process./RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta). in WP don’t mean the same thing they mean everywhere else. It’s confusing to any developer who’s starting with WordPress having worked on other projects before.

The process outlined in the post makes sense and I think it would be a great improvement. Being consistent with the industry-standard processes will make WordPress less confusing to new contributors 🙂
Tonya Mork 4:48 pm on November 28, 2020
Thanks for the proposal Francesca!

Defect work scheduled during BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. is risky. Even a minor fix could have unforeseen impacts.

I do support the proposal of reserving Beta to increase the quality of each release.

However…

A significant amount of defect (bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority.) work happens after feature/enhancementenhancement Enhancements are simple improvements to WordPress, such as the addition of a hook, a new feature, or an improvement to an existing feature. freeze. Currently, this defect work happens during Beta. A further adjustment is needed in the proposal to ensure focus is not lost on this defect work. Why? To mitigate the risks of
- releases with less and less bug fixes
- bug fixes being kicked months down the road
I concur with Timothy, Sergey, Garrett, and Andrew. Each deserves their own focal areas within the development cycle to ensure each gets prioritization and bandwidth:
- Feature/enhancement work
- Defect work
However, there are 2 different proposals presented of how to allocate the defect work:

Proposal 1: feature freeze and then do defect work

This proposal is for a feature freeze 2-3 weeks before Beta 1 and then allocation of this time period for defect work:

Per Timothy:

If we don’t want to make any changes in beta, then we should freeze feature development 2+ weeks before beta to grant contributors this time.

Per Andrew

Thinking it’s a good idea to have a “Feature freeze” cutout time, just lets not call it “Beta” 🙂 Two-three weeks before beta-1 sounds like a reasonable timing and would match the current workflow.

This proposal shifts Beta downstream by 2-3 weeks to make room for the defect work.

I like this proposal as it has clear definition in the major releasemajor release A release, identified by the first two numbers (3.6), which is the focus of a full release cycle and feature development. WordPress uses decimaling count for major release versions, so 2.8, 2.9, 3.0, and 3.1 are sequential and comparable in scope. cycle after the major release is kicked off and when the release squad is in place.

However, I have some concerns:
- It does not capitalize on the overlap between releases.
- It does not provide a way to reduce the overall release cycle.
- Does the major release squad need to be focused on the continuous defect work (i.e. defects not directly caused by the release)?
Proposal 2: Defect work during release overlap

This proposal front loads the defect work to overlap the previous release’s Beta and RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta).:

As Sergey notes:

As long as we branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". at Beta 1 to allow for continuous development, where the branch is reserved for regressions and trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. is open for any commits to keep the momentum on tickets that were punted

I like this approach.

Why?
- Work continues with purpose and focus while the previous release is in its testing and release phases. It capitalizes on the time between major releases while keeping the momentum rolling forward.
- It provides an opportunity to shorten the overall major release cycle.
However, this approach creates a new challenge (or opportunity) as we currently do not have a release squad ready to go during this overlap period. Though I wonder…
- Do we need the release squad to be involved in this early phase or even in the defect cycle?
- Could the component maintainers and CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. Triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. Team work together to help prioritize and escalate defect work?
If yes, then the major release squads can focus all of their energies on scoped feature and enhancement work. Hmm…

Questions

A few questions come to my mind….

Is Beta going to be long enough?

Beta serves multiple purposes:
- To validate and test the release itself in a broad mashup of environments
- For the community to validate and prepare their products and services for the coming release
With a shortened Beta, is this enough time for the community? How much time does the community need?

What about blessed tasks?

For blessed tasks that alter core’s production source code (i.e. not tests or build or test tools), how do we mitigate the risks of introducing a bug or breaking change after Beta 1? Do we need a new classification to differentiate?
- Josepha 12:17 am on December 2, 2020
  
  I second all the risks and questions being raised here.
Josepha 12:58 am on December 2, 2020
Generally, I’m always open to experiments that we think will improve the experience of users, contributors, and the WordPress community as a whole. This one looks like it has two big parts: align naming conventions with industry standards (including the definition of the named phases) and decrease the time between betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. 1 and the release date.

I have a few concerns and a few gimmes for both.

Rename and redefine
- Gimme: I think renaming our phases is fine. We have a lot of new contributors and anything that makes finding their way easier is a good change.
- Concern: I worry that we aren’t allowing space for older bugs that aren’t specific to the planned features in the release. I also worry that by calling hard freeze earlier in the process we narrow the window for feature inclusion too much. I think Matt agrees with my thoughts there, as well.
- Concern: I don’t like limiting ourselves to feature specific bugs right now, since that excludes so many of our volunteer contributors. It’s harder to work on features since they are complex and fast-moving, and older bugs present more opportunities for casual contributors.
Shorten Beta and RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta).
- Gimme: Streamlining process (in order to get shorter beta/RC) means better processes for contributors. That feels like a win for CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. maintainers and Core component maintainers.
- Concern: We already have feedback that it’s hard to keep up with changes for our pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party/theme authors. While a change like this is possible, it would require some paradigm shifts that I don’t think have been fully explored.
- Concern: Regarding when we open alpha, if Beta and RC are both shortened, then I’m not sure we end up with a net gain of time when compared to our current, longer RC. It seems that the people who work on early alpha tickets are the same as those who help us get across the finish line. I think if we split their attention we would either have a worse product per release, or very burned out contributors—both of which I want to avoid.
- Francesca Marano 7:27 am on December 2, 2020
  
  Thank @chanthaboune for chiming in.
  What’s a “Gimme”?
  - Josepha 3:35 pm on December 2, 2020
    
    A “gimme” is an easy win. Excellent question and apologies for the jargon. 🙂
- Joost de Valk 7:37 am on December 2, 2020
  
  I worry that we aren’t allowing space for older bugs that aren’t specific to the planned features in the release. I also worry that by calling hard freeze earlier in the process we narrow the window for feature inclusion too much.
  
  I think we should just branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". off earlier, and keep trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision. open for normal business. That way, *everything* can be worked on all the time, but it won’t be included in the next release depending on when you commit it. That’s fine, every piece of open sourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. software I know in the world works like that, except for WordPress.
  
  It also relates to your other concern: “We already have feedback that it’s hard to keep up with changes”. It’s hard because right up to RC1 and sometimes even within RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta)., new stuff lands. If the new stuff landed in trunk instead, and from betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. onwards we’d only have bugfixes on the release branch, we wouldn’t have so much of a moving target.
- Aaron Jorbin 9:34 pm on December 2, 2020
  
  I agree with all of what Josepha is bringing up. Thank you.
  
  Are there any stats of how many bugs have been introduced during betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process.? Is this aiming to solve a real problem or a hypothetical one? Some data would be really helpful.
  
  I have concerns about shortening the RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta). period as that is the time that sites/plugins/themes really get ready for the upcoming release. In my eyes, anything less than 4 weeks here is insufficient as organizations that do two-week sprints need a bit of lead time to get a ticketticket Created for both bug reports and feature development on the bug tracker. groomed and into a coming sprint and depending on how their sprint cycle works.
Beau Lebens 2:12 am on December 3, 2020
From an “external” perspective (I’m coming at this from the point of view of WooCommerce, which obviously relies entirely on WordPress for stability), I’d endorse anything that gives us a clear target for testing against, and ensuring compatibility with (preferably for “as long as possible”). One of the challenges we’ve faced is when major functionality is merged in late, and then there are a bunch of seemingly-benign bugfixes during betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process., which turn out to have significant impact on either our coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. software, or our extensions (plugins that rely on WooCommerce itself).

So from our perspective, anything that makes it clear when we can really dive in and test our ecosystem thoroughly/deeply, and trust that there won’t be any significant changes from then on (in that release), would be really helpful.

In case it’s useful, we’re do something like this;
- we release monthly, on the second Tuesday of every month
- just under a month before release, we do a feature freeze, where we stop accepting new features/major changes, and switch to getting everything merged, verified, etc.
- 3 weeks before release we tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) our first beta (into a separate branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch".) and go into a pretty heavy _internal_ (within Automattic) testing/fixing mode where things are still pretty fluid, but we’re not adding new features at this point. We will periodically release additional betas if there are a number of fixes happening in this period. Work can continue in “trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision.” at this point, since everything related to the release itself is in a separate branch. If needed, we also cherry-pick fixes.
- 2 weeks before release we tag a release candidaterelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta)., which is what we consider the best testing and integration candidate for external (outside Automattic) developers, extension builders, people who want to get a head start testing on live stores, etc. During this period only critical issues are addressed.
Francesca Marano 9:02 am on December 3, 2020

Thank you everyone for taking the time to reply!
I will recap the comments and publish them here as a separate blogblog (versus network, site) post before the end of the year.