Estimates are not metrics

This post has been inspired by the countless arguments conversations I have with my good friend David Whitney who has produced his own post about estimation that should also read: https://www.davidwhitney.co.uk/Blog/2021/02/10/agile_software_estimation_for_everyone

Estimation is an intrinsic part of software development, it is however, used wrongly in my opinion. Within a small iteration of a development team, the act of estimation itself is an important tool to understand complexity, perform slicing, share understanding, the “number” that may be produced as part of that is less important. In contrast, when you’re trying to establish a Time to Market, or a budget, the rolled up “number” is important. In this post I’m going to discuss what is wrong, and how to frame this with your teams, managers, leaders.

I’ll state this so it’s not ambiguous, I HATE story points. I believe the only unique value they provide is a stick to beat the team with when velocity changes, or they get the number wrong. Every other perceived value can be achieved much easier, and with more advantages, using other means.

Further clarification, if you arrived here with an anticipation that I’d give you a roadmap to remove estimation from your company, move to a “NoEstimates” culture or provide a reference for “Martin said estimation is bad” you’re in the wrong place. NoEstimates doesn’t mean we don’t estimate, it’s just a hashtag that Woody created to start a conversation about the use (and misuse) of estimates in Software Development with like minded people, and how we can do better.

What is “Estimation”?

So what exactly are we referring to when it comes to “Estimation” in software development.

Lets start with estimation as a term in general vocabulary. Estimation isn’t a software development concept, we perform all sorts of estimation in our daily lives. From estimating the impact of rush hour on our travel to work, to how much we’ll need to save each month. We do all of this without really thinking about, and without complaining, and the reason is that we know “why” we’re doing it.

According to Dictionary.com:

to form an approximate judgment or opinion regarding the worth, amount, size, weight, etc., of; calculate approximately
Estimate | Definition of Estimate at Dictionary.com

Essentially an estimate is a “guess” at a specific measure of something based on information you’ve been given. Let that sink in, it’s a GUESS. It’s sometimes influenced by analysis, or investigation on the subject, however, it’s still a guess.

Most importantly, an estimate isn’t just 1 thing, it has “dimensions” such as size, weight, duration, value, etc. Therefore in order to know which dimension we should use, we need to know how our estimate is going to be used. Why do we estimate how long it will take to get to the office in rush hour? it’s to work out when we need to set off, therefore estimating the number of miles we’ll travel around the back streets of London to get around the traffic won’t necessarily help.

Estimating in Software development is no different. Asking a team to come up with arbitrary numbers (story points, t-shirt sizes etc.) are pointless if they don’t know why, and how they’re going to be used. Further, story-points, t-shirt points hold up better at larger abstraction levels than smaller ones.

Bad estimates

The things I’ve seen as what I would call “Bad Estimates” can normally be categorised as estimates that are subsequently used as “metrics” that are monitored. David Whitney defined estimates as “like a shotgun, accurate at close range, and useless at long range”, and this rings true when you start to think about estimates as a measure. If they aren’t accurate beyond the next few iterations, they can’t (and shouldn’t) be monitored beyond that.

In short, don’t take those story points, multiple by 1.5, put that in a Gantt chart and beat the team when they’re not at the right milestones based on those guesses they made. This practice has, over time, forced developers to be over cautious, non-committal, and generally push against the estimation process in general. Resulting in everything from overly conservative estimate “It’ll take 2 years at least”, or a complete refusal to engage.

Estimation is a discovery task

One thing that myself and David agree on is that the act of performing estimation at various levels is an important part of discovery. If this is done with the team doing the work, and facilitated in the right way, you can get accurate ranges at high levels of abstraction.

Performing the estimation work will help the teams understand lots of things about the work being proposed including:

• Complexity of the ask (what specialisms might be needed, Automation, performance, compliance)
• Slicing (how can we split this up and deliver in smaller chunks)
• Dependency (internal or external things that will dictate when it can be completed)
• Spikes/Investigations (do we need to run smaller discoveries to prove something)
• External factors affecting delivery (Certification, procurement, etc.)

All of these are incredibly valuable to the software development process, and are an important by product of estimation. They can help with the mental wellbeing of the team by ensuring that small victories can be celebrated, and ensure that the team doesn’t over commit in smaller cycles.

Outcomes not Outputs

When Estimates become metrics, you’re actually measuring the output of your team, and not the outcome. That is to say that you’re not measuring the “ask” of the team, but rather metrics that are by products of that in the hope that they’re indicative of the outcome you wanted.

When you move into a product business with product teams, the focus is no longer on the output of the teams, teams are autonomous and measured by the outcomes they’re asked to produce. In these teams, it’s simply not acceptable to say “but we have a predictable velocity”, or “we did all these story points though”, these just aren’t how the teams are measured anymore. The team will test, learn, adapt, and focus on constant feedback with regular value delivery.

In these sorts of teams, the value of an overall estimate based on time is at best irrelevant, at worst detrimental to the teams focus and goals. This is where it becomes important to think about the reason for estimation, and establish what the end goal of the estimation is.

Making Estimation work

Different types of estimation WILL work for different usecases. There is never a one size fits all approach to providing any kind of estimate.

Estimation can work in Software Development, and the key is knowing “Why” you’re either making estimates, or providing estimates to others. This is what allows us to provide the right “kind” of estimate, at the right level, and also inject the right amount of effort into providing the estimate in the first place. These are the sorts of questions you should be asking:

• What will it be used to inform? (Project Budgets, Resourcing, Customer commitments)
• What’s the risk of it being too long/short/high/low/expensive/cheap? (Project green lights, regulatory, reputation)
• What does success look like for the estimate? (Delivery time, Cost, Profit)

What we’re trying to establish is the “value” of the estimate we’re giving, so we can apply the appropriate amount of time and effort to producing it.

Monitoring the estimate

Once you’ve provided an estimate on a large project, you’ll often find that people then want to use that to “track” progress against that “guess”. This is where estimates become toxic. This is specific to Large projects that span multiple months/business cycles.

Estimating the cost/timeline is providing an endpoint of the project. This is the definition of success (or failure) of the project. This doesn’t mean that you can use this to monitor whether a project is on track on a regular basis.

When you start to question the stakeholders for the reason the monitoring, what I’ve found is that what they really want to know is “Are we on track to deliver the project based on what was projected”. I’ve never found that monitoring on a granular level can give the right level of confidence to the stakeholders.

So what’s the answer? Milestones. A milestone is a point in time that you can use to check whether you’ve completed the required things that will get you towards that outcome. These don’t need to be “finished” things, they shouldn’t be a “number of stories” or an “amount of points”, they should, however, give you an indication that you’re on the right road, and you’re heading in the right direction, at the right pace. I would also encourage you to see these not a Date (i.e. 12:36pm on March 25th), but more a vague range like “End of March” or “Before the summit”.

Breaking up a larger project into a series of milestones that provide tangible outputs is BY FAR the best way to give stakeholders the confidence that the project is heading the right direction. These milestones should have defined outcomes that are tangible, demoable, and show a progression to the overall outcome of the project.

Defining the milestones is something that you should do as a collaborative task with the WHOLE team, and include the stakeholders. If the stakeholders don’t want to be involved in that, they likely only care about the end goal, and therefore the monitoring isn’t really important.

There is literally know better way to give the stakeholders the confidence that you know what you’re doing, and you’re on time/budget. There is no amount of numbers, spreadsheets, Gantt charts or (god forbid) powerpoints, that can replace actually talking to people regularly, and showing them the things they asked for.

Conclusion

To summarise my stance on this:

• Large project estimation is important, and relatively easy to achieve with a degree of variance.
• Monitoring of the progress of Large projects should use different information than estimation.
• Estimation shouldn’t be used to produce a number that is tracked

In general, moving to outcome tracking on smaller iterations is where I desire all my teams to be. This is all about empowerment and trusting the teams to deliver value than work to metrics that don’t map to business value.