The Tyranny of Velocity: How not to measure Agile Team Performance

Velocity is one of the, or THE, key metric used by most agile teams to size, plan, and ensure that they are on track. For scrum-based teams, it is hard to think how they would operate without it.  It’s simple, easy to track, flexible, awesome….right?  

There is a dark side to velocity, however, that we must be careful of.  Instead of being a tool used by the team, it can become something used to control the team, intentionally or not, and damage its primary purpose.  This is the Tyranny of Velocity.  

The Source of the Problem

It all begins innocently enough when organization want to show progress towards agility, better understand the team productivity, to enhance predictability, or to better understand how teams compare to one another.  

Velocity seems like an ideal metric for all these things. The team is already using it; what could be the harm?  

The harm begins with Goodhart’s Law – “When a measure becomes a target it ceases to be a good measure. “ 

The simple act of turning a measure into a goal is enough to subvert our intentions of it. So even if it is not your intent to utilize velocity as a control mechanism, over time, it is unavoidable as the intent will get subverted.

Velocity should be a measure used by the TEAM to manage their work and to indicate progress on a product iteration.  By turning it into a control metric we subvert its use by the team and in fact can negatively impact the very things that we are trying to use it to measure. As a control mechanism, someone other than the team has too much control subverting self-organization & ownership.  

On top of this, there a better metrics to be used for measuring productivity and effectiveness than velocity such as throughput/flow or value metrics.  Focusing on velocity leaves out a lot of things that impact productivity that are not included in velocity and bring too much focus on the team as the key source of inefficiency leading to mis-optimizations. 

Furthermore, overly focusing on velocity leads things not getting done, that should be done, such as refactoring, which typically leads to a degradation in the health of the product and eventually a reduction in cycle time or agility. Velocity is not a system metric and thus can only give us a limited view without context.  We should look for system metrics that align to the overall goal of producing value from the system and not a single optimization point. 

There are some warning signs that you can look for to see if you are mis-using velocity, or may have introduced other command and control relics, to inappropriately influence team behaviors. Any of the following warning signs suggest that something is wrong, and it is likely connected to how you measure:  low innovation, and lack of emergent designs and emergent architecture, little continuous improvement, with low use of refactoring, low reuse, or slow adoption of new techniques and standards.  These things can further be exacerbated where in addition to velocity being externally regulated, we have teams with a lack of ability to say “no” to users’ stories that are not ready to be worked.

Moreover, teams should be cautious about the high rate of stories that are not completed in a sprint or that never make it to production. The current state of the industry is far below where it should be on these metrics.

Culture - Trust / Commitment

To make things worse, agile or growth cultures, depend on trust and safety whereas the productivity and control metrics are run contrary to this. In many organizations the agile teams are in a perpetual state of stress leading to a non-sustainable situation.  When there are productivity problems in agile, they are seldom the “fault” of the team and more attention should be brought to the system, to the environment, and to management.  Again, system metrics are better for identifying these things.  

Team commitment is another critical pillar that is often eroded in that organization that utilize velocity as a control mechanism when velocity is used to measure productivity, it can alter what that commitment means. Team may feel compelled to commit or change how much they commit to base on external pressure or velocity and not on what they feel they can get done.  This is exacerbated in organizations where the initial estimate is done by some other group other than those committing to it.  

Impact

There can be a significant when the use of velocity become tyrannical.  Most chiefly, we see a lack of obtainment of expected agile benefits leading to with reduced throughput & business value. In addition there is almost always an increase in technical debt as teams cut corners or stop including things in their estimates in order to keep up velocity.  

We often end up with a poor balance of effort between features development and other things such as refactoring, resilience, performance, and maintainability.  In addition, we see slow adoption of new techniques, automation, technologies, etc.  Collectively, these impacts often lead to unengaged developers.

What to do about it

• Do not use / report velocity directly outside the team (product owner is part of the team) since it can be misinterpreted/abused.
• Keep using velocity but use it primarily as a team level tool for coordinating work.
• Select better metrics to measure throughput, cycle-time, etc. that look at the entire system.
• Help teams to ensure that they include refactoring, continuous improvement, and other long-term health items in their estimates.
• Do survey product teams for satisfaction, as an indicator if team performance is changing.
• Do survey team satisfaction as impediments to productivity will tend to show up there
• Do ensure that your continuous improvement cycles / retrospectives are generating leading to changes on a frequent basis.
• Don’t include partially done items in the velocity, no partial credit.
• Ensure all work is estimated, included in the planning, velocity metric including ad hoc, defect, small request, etc.  Some organization don’t include this which affect numbers.

The article has been written by Gregory Wittenbrook, VP Transformation Services, Mphasis