Jonathan Desrosiers 3:35 am on January 4, 2019

Short Term Trac Milestone Ticket Triage Proposal

Due to a handful of different factors, the number of tickets in unreleased milestones on Trac has become unmanageable. One of the main reasons is the following scenario. Person milestones ticketticket Created for both bug reports and feature development on the bug tracker. for the next release with good intentions. Ticket is not quite ready or release scope/priorities are changed, so it gets punted. New ticket is added to next milestone. Repeat.

This, in addition to the longer than expected timeline for 5.0 and the 8 4.9.x releases, has caused a snowball effect where a large number of tickets have continued to accumulate and get passed to the next upcoming milestone.

At the time of publish for this post, there are 473 open tickets in the 5.1 milestone and 55 open tickets in the 5.0.3 milestone (most of which will be punted to 5.1 before the 5.0.3 RCrelease candidate One of the final stages in the version release cycle, this version signals the potential to be a final release to the public. Also see alpha (beta). is packaged on January 7th).

This is confusing for contributors, making it difficult to prioritize what to work on or understand what project goals are. It also multiplies the burden on release leads. The more tickets there are, the more bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. scrubs and punting decisions required.

At some point, a reset button needs to be pushed to cleanse upcoming milestones of tickets that will not realistically be completed.

Proposal

This two-step process is being proposed to prevent punting 500+ tickets to the 5.2 milestone when 5.1 is released.

Part 1

Component maintainers and ticket owners/assignees should scrub their tickets in the 5.1 milestone. When evaluating each ticket, the following factors should be weighed.

Is the ticket still valid? If not, close it with a note.
Is it a bug fix or general maintenance task?
Does it fit into the 9 Priorities of 2019?
Can it realistically be tackled in the very near term?

After evaluating the ticket, discretion should be used to move the ticket to the 5.1.1, 5.2, or Future Release milestones. If the 5.1.1 or 5.2 milestone is chosen, a ticket owner must be assigned to ensure the ticket receives the needed attention.

The ticket owner is responsible for knowing the status of the ticket and ensuring it is either ready in time for the milestoned release or punted in a timely manner if it not. This person does not have to be the only person working on the ticket, they will simply serve as the lead for that ticket.

There is also no shame in unassigning a ticket when moving to the Future Release milestone. This will make way for another contributor to take ownership.

Step 2: Bulk Edit

Once WordPress 5.1 BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. is released, all remaining tickets in the 5.1 milestone will be moved to the Future Release milestone. The only exception will be tickets that have been reopened (each of these should be evaluated more closely). This will stop the snowball effect, allowing the 5.2 milestone to remain small and focused only on the priorities determined by the release leads. Tickets can always be moved to a milestone when deemed appropriate and an owner is identified.

Long Term: Triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. Team

While this plan only addresses the very short term, an effort is underway to establish a Triage Team as one of the 9 Priorities for 2019 to address the larger issue of the over 6,500 open Trac tickets. @chriscct7, @karmatosed, and myself (@desrosj) are working on a rough draft set of rules for triaging tickets as an initial proposal for the foundation of the new Triage Team. Look out for this in the coming weeks.

Feedback

Questions? Concerns? Suggestions? Please share them in the comments. This purge needs to happen, but this is not a final proposal. Feedback is welcome!

JJJ 3:15 pm on January 4, 2019

I just got an email from TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. that dozens of tickets were resolved as wontfixwontfix A resolution on the bug tracker (and generally common in software development) that indicates the ticket will not be addressed further. This may be used for acceptable edge cases (for bugs), or enhancements that have been rejected for core inclusion.. Is that part of this?
#6331, #6619, #11049, #14364, #22601, #24552, #24675, #24865, #25070, #25546, #26332, #27076, #27148, #27530, #28070, #28413, #28566, #29359, #29386, #29643, #30187, #31156, #31219, #31229, #31396, #31479, #31642, #32273, #32513, #32554, #32583, #32638, #32777, #32803, #32912, #33249, #33584, #33876, #34007, #34181, #34188, #34230, #34254, #34355, #34792, #34882, #35158, #35285, #35299, #35334, #35397, #35802, #35932, #36216, #36223, #36312, #36469, #36474, #36479, #36483, #36540, #36575, #36619, #36683, #36689, #36734, #36756, #36840, #37091, #37313, #37482, #37577, #37758, #37805, #37829, #37862, #37970, #38055, #38115, #38120, #38509, #38784, #38834, #38934, #39271

It was suggested in the ticketticket Created for both bug reports and feature development on the bug tracker. comments that if a ticket is still relevant, to reopen it. There’s not really an easy way to go through this many tickets at once to cherry-pick the ones that are worth pursuing.

I looked at a few of them, and they all looked like bugs that still exist. I also couldn’t find any relevant SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. chats where these ticket IDs were brought up to be bulk closed.

It seems, to me, this type of clean-up effort unintentionally creates more work, because in addition to the bugs not being fixed, people need to close them, people need to resurrect them from the dead, people need to prove the issue still exists, and people need to lobby the crowd to prioritize fixing them – if all of those things become standard affair for creating a simple bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. report, people will simply stop reporting bugs because they require more attention than they can give them.

I’m all for bringing the numbers down, but I’m also worried we are inventing more work for people who already have more to do than is getting done.
- Jonathan Desrosiers 3:21 pm on January 4, 2019
  
  This is completely unrelated! It appears that almost all of those tickets are in some way TinyMCE or Editor related. I think that the idea here was to close all old tickets related to those in an effort to surface issues with the new editor.
  
  That came up in our early discussions as a potential way to triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. old tickets that may no longer be relevant, but we have not acted on anything as of yet. The plan above would be the first bulk action by the Triage Team.
  - Jonathan Desrosiers 9:36 pm on January 9, 2019
    
    Wanted to post this here so that someone reading this discussion in the future would be able to connect the two events. The above bulk edit that @johnjamesjacoby details was followed up with a second bulk edit that touched significantly more tickets (over 2,000). This bulk edit was so large that the server did not send out notification emails.
    
    This bulk edit was also unrelated to the Triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. team or the above proposal.
Andrea Fercia 3:59 pm on January 4, 2019

Re:
> Once WordPress 5.1 BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. is released, all remaining tickets in the 5.1 milestone will be moved to the Future Release milestone

Not sure it’s the best path forward. That would just move the work load from the release leads to the component maintainers 🙂

At least in the accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) team, but I guess this applies to all the teams, we regularly do bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority.-scrubs against the “pending review” and the “future release” report.

Moving tickets that have already been milestoned “back to future release” is basically a step backwards in this flow and will force teams to bug-scrub again the same tickets.

Not sure what’s the best alternative, but I’d like to avoid double work 🙂 Can’t they just be punted to the following release? I think this would still help in stopping the snowball effect, just a bit more slowly maybe?
- Jonathan Desrosiers 4:53 pm on January 4, 2019
  
  Thanks for this feedback, @afercia! I think what you described in the last paragraph is the snowball effect, though. Punting all accumulated tickets to the next milestone.
  
  This plan does add some burden to the component maintainers. But, it is the responsibility of component maintainers to ensure that their tickets stay in appropriate, realistic milestones. Over the last several releases, when a milestone is released, tickets that did not make it have just been punted to the next release, and this has caused the problem. While the a11yAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) component is very diligent about bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. scrubs, some are not.
  
  For some context, there are currently 49 accessibility tickets in 5.0.3 and 5.1 that meet the criteria for punting to Future Release defined above. Of those, only 30% (15) have owners, and less than half (48% or 24) have been updated in the last 3 months.
  
  The reason moving the tickets to Awaiting Review is not being suggested is because the ticketticket Created for both bug reports and feature development on the bug tracker. has already been sanctioned as valid and worthwhile by being milestoned. Future Release maintains that distinction. We also want to give a bit of time for component maintainers and ticket owners to move their short term priorities into the 5.1.1 or 5.2 milestones before performing this update.
  
  At the time that these tickets were placed in a milestone, it made sense. But contributor availability changes, project goals change, and it’s OK to unassign or unmilestone tickets. They have not been receiving the attention they need to land and need a second look to reevaluate the priority and whether tackling them short term is realistic.
  - Andrea Fercia 5:36 pm on January 4, 2019
    
    > add some burden to the component maintainers. But, it is the responsibility of component maintainers to ensure that their tickets stay in appropriate, realistic milestones
    
    “some burden” is a bit optimistic 🙂 I can speak only for the accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) team: many of those tickets were initially milestoned for 4.9.9 that was then cancelled. So, the punting effect involved more tickets than usual.
    
    Please consider the accessibility team is not responsible for all the tickets that were punted and milestoned to following releases, someone else did it 🙂
    
    No one is perfect, and I’m sorry if some tickets have no owner. I’m also sorry if some patches haven’t been updated in the last 3 months, as you may know the accessibility team is a very small team and the only person who codes something was busy on GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ stuff.
    
    Regardless, I wouldn’t have objections if the number of tickets moved to Future Release was relatively small. Instead, moving a huge amount of tickets to Future Release basically forces all the teams to double their work, in most of the cases for reasons they’re not responsible for. I’m sorry but that’s a bit disrespectful for our time.
    - Jonathan Desrosiers 6:08 pm on January 4, 2019
      
      No disrespect was meant, and I’m sorry if it came across that way. My point in the exercise above was only to illustrate that there is a large number of tickets that continues to be punted, not to try and say that the team has not been doing work, or that they are too small. This was posted so a discussion could be had about what works for everyone with the understanding that what was proposed above may not be a policy that works for everyone.
      
      That said, I totally understand that there are several factors outside of any component maintainer’s control (`trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision.` being frozen, 5.0 focuses, etc.). And I also understand that accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) is a focus and not a component on track, which adds a bit of complexity because the tickets are spread out among many components.
      
      How about this as a compromise:
      
      Reopened tickets, per above, will be considered on a case by case basis.
      
      Accessibility tickets marked `commit` (4) will be moved to the 5.1.1 milestone if not committed for 5.1.
      
      Accessibility tickets with owners (14) are punted to the 5.2 milestone.
      
      Accessibility tickets modified after December 1, 2018 (13) can also be moved to the 5.2 milestone.
      
      The remainder of the tickets (18) are punted to Future Release.
- Jonathan Desrosiers 4:57 pm on January 4, 2019
  
  Also, I don’t want it to seem like I am trying to force the above through. Definitely willing to hear suggestions, but I don’t think bumping to the next milestone is a good alternative. That is what we are trying to avoid.
  - Andrea Fercia 10:33 pm on January 5, 2019
    
    You may not know, but the accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) team made some extra bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. scrubbing sessions and doubled their efforts to milestone for 4.9.9 a good number of small, self-contained tickets that could have been reasonably addressed. 4.9.9 was than cancelled, and those tickets punted by other persons to other milestones or Future Release. The situation is now so messed up that I wouldn’t know where to retrieve the list of those tickets.
    
    A lot of time and work was just wasted, generally this doesn’t makes people so happy.
    
    Reducing the number of accessibility tickets that get punted could help, thank you for your proposal.
    
    Instead, regarding the larger triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. plan, I think any proposal should be discussed during dev chat, as it impacts all the teams and it’s unfair the decision gets made by just a few persons. Also, what happened yesterday with the batch-closing of ~2300 tickets is unfair as it came out of the blue with no public discussion (I may have missed it). See also all the concerns expressed in that regard by many on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..
    
    I’d also like to point out the “2 years of inactivity” criterium for closing the tickets isn’t the best one, as all the teams were forced to stop any activity not related to GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ for more than one year due to the 5.0 release.
    
    So basically everybody was told to wait. We (we as in: many committers and components maintainers) diligently waited for more than one year and, to be fair, many of us were not so happy for that. And now many of the tickets we were waiting to fix gets just closed? 🙂
    - Jonathan Desrosiers 9:28 pm on January 9, 2019
      
      Sorry, I had missed your follow up here.
      
      I understand and share some of your frustrations regarding the batch-closing of tickets last week. These were two completely unrelated things that coincidentally happened at a similar time.
      
      The Triagetriage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. Team will be fully transparent in all bulk actions that are proposed (hence this proposal). Ideally, the right processes can be outlined that work universally across components and empower component maintainers and gardeners to more effectively milestone tickets to the point that bulk actions will be very rare moving forward (especially since they are not adequately communicated in the email that is sent out).
      
      The right triage processes should not create more work for any component.
Michael Nelson 5:14 pm on January 4, 2019

👏 IMHO it sounds unmanageable to keep punting nearly 500 tickets to the next milestone. They can’t possibly all go into the next release so it’s inaccurate to label them like that. “Future releases” sounds appropriate.
rweil55 4:15 pm on January 7, 2019

For the bulk edit, and future releases: Rather than pushing every ticketticket Created for both bug reports and feature development on the bug tracker. to “future release” would you consider an alternative based on how long a ticket has been around and/or how important someone thinks it is?
One semi-automatic criteria for that might be:
Push to “next milestone” those tickets that have only been in one release cycle.
Push to “future release” tickets that have been in two or more release cycles.
If it has been sitting around for several cycles and no one has dealt with it, can the ticket be assumed as not a major issue.
A rule like this could be applied for each release to eliminate further accumulations.
Whether the number is one, two, or three might be determined by how the split works out.
Just a thought
Roy
- Jonathan Desrosiers 4:46 pm on January 17, 2019
  
  Thanks for this comment, @rweil55! Sorry I missed it when you first left it.
  
  I think what you are describing is what typically happens when tickets are scrubbed manually. Contributors evaluate each ticketticket Created for both bug reports and feature development on the bug tracker. and determine if it needs to be placed in Future Release because it is not getting worked on or not close enough for the next release, or, the next milestone if they are close but just couldn’t get over the threshold for committing.
  
  Fortunately, since this was written, it has proved to most likely not be needed. A handful of contributors have been scrubbing all tickets in the 5.1 milestone. Using the logic you described, they have been choosing one of those two paths.
  
  In general, the feedback on bulk edits has been that they are not favorable. But, I think what you outlined above is reasonable to consider for any future bulk editing related to milestones.
  - rweil55 7:07 pm on January 17, 2019
    
    The secondary thought here is “applied for each release to eliminate further accumulations”.
    It is great that this pass is being done manually, because of the backlog and previous non-notification to submitters. But one should consider an automated process that is known by all for future releases.
    This may eliminate some future bulk edits, and the grousing that occurs on a change in procedure. It would eliminate the complaints that might arise over perceived unfair assignment.