Correct Estimates? A personal perspective

Motivation

A while ago I paused an exchange on twitter. I have been in a strong disagreement with what was stated from Alex. To avoid being bullish on a short reply, it took some time to reply in due respect and also purposefully.

Nevertheless, I need to agree to some extend: There might be cases or domains with a lot of routine work. Then you have evidence to provide very accurate forecasting.

But in my domain I need to state that such correct estimates are rather an anomaly. Reason is complexity and novelty of constraints or requirements. But we try to estimate as educated as possible and reasonable in the context given.

Also at estimation time, no one can claim correctness on an estimate as it is prior to implementation (and integration testing) of the requirements related.

Asking for correctness of estimates can lead to two major issues:

Overinvesting in investigation of all circumstances and details
Requesting correct estimates might lead to add high buffers on each estimate. That allows proving correctness by being slow to meet the exact efforts later.

A counter strategy against both symptoms might be: Implementation based on less accurate estimates, what is a cheaper approach to gain prove on real efforts. Consider what buffers sum up if you think about a portfolio of scope items.

But let’s consider estimates and estimations and the use of them deeper, in a broader context.

Exploring a single Estimate

Before we dive into further evaluation, we need to gain a common understanding on a few basic terms.

Precision & Accuracy

What counts for me in general is not asking for correctness of an estimate. I rather need to understand the precision and accuracy of the estimates I get. Plus how estimates differ on specific teams or on specific areas.
These could be front end development, conceptualization, complex algorithm, micro service development etc.

Above picture introduces the terms precision and accuracy quite well

What you might need estimates for

To make good estimates, it is also key to understand what you need them for. A comprehensive but not complete set of situations might be: Make a first guess on ball park figures of investing in a project idea

Initial assessment with a customer on a time and material project
Committing to a fix price fix time project (with the risk of penalties)
Building a roadmap over a period of 6 month
Release planning over 2 month
2 week sprint planning
Ad hoc disaster recovery of critical infrastructure

All for either your own planning, being able to commit to stakeholders or to sign contracts or change requests.

Granularity of requirements

What you also need as an input for estimates are requirements, constraints and dependencies as well as context to the estimating party.

Understanding the right level of granularity of the requirements and above mentioned information is paramount. The granularity can be limited by three dimensions:

What level of information is present
How much effort do you want or need to invest into the estimation process
A first sanity check prior to estimates itself needs to be: Are you set to do the estimate as wanted or are there gaps in prep work?

Repeating myself partly, you need to understand the purpose you need answer.
What needed to be clarified before hand:

in what phase of the project you are,
what you need the estimate for and
how much effort you can afford in estimation as well as
considering your domain or business context.

Above checks allow you to gain an understanding on the right level of detailing needed by the requirements, and breaking it down in related bits and pieces, level of understanding dependencies and so on.

As a control question on having the right granularity you can check roughly:

Do I want to have requirements that can be dealt with by a team of 3 in a month, or
by a single contributor in a week or
by a developer in less than a day!!

Scope Item as the subject of an estimate

Before getting started. you need a well defined thing to apply your estimate on. I call this the Scope Item.
The scope item is the thing that needs to be delivered at the end.

Directly associated with the scope item are

Requirements – known or unknown ones
Dependencies – with other scope items, technical assets or interfaces, organisation or commercials as well as time constraints or documents
Context and
Risks – might it be unknown scope or dependencies, moving context or that you miss well established skills or resources to implement

More loosely coupled but quite important are

The person or entity requesting the scope item to be delivered
Assumed business value
Overall schedules or roadmaps creating a sense of urgency or constraints
A potentially already known delivery team
External parties needed to solve dependencies related

The scope owner (or Product Owner)

The owner or representing and authorized role that is obliged to potentially explain, negotiate or compromise on any of the direct associated matters mentioned above, having authority to prioritize and to negotiate if needed with the estimating and delivering parties is a core role.
Note: Pure document driven scope and estimates without a role that is accessible during estimations are a risk in itself, as ambiguity or conflicts between areas of scope are often leading to confusion, loss of time and suboptimal and less accurate estimations.

Why and how to use Story Points as estimation scale

Expressing that I do not want correct estimates and not misleading the estimating party, I am starting estimation via Story Points – or Fibonacci sequence as alternative.

0, 0.5, 1, 2, 3, 5, 8, 13, 20, 30, 50, 100, ?

Story points usually express complexity of requirements, but not efforts. And they are abstract numbers, but with growing distance from each other the larger they get.

The growing distance between growing numbers demonstrates implicitly:
Estimations are getting broader and fuzzy in accuracy the larger the estimate – and hence the requirements are. And it keeps the estimating party from starting “bean counting” but rather understand what accuracy is reasonable to consider.

To stay in reasonable ranges, I often ask teams to break down scope items (the thing getting estimated) when they are delivering estimation results over 20 or 30. This needs the analyze, product owner and estimation team to ad hoc break down and maybe detail requirements in their estimation session.

This said…

I do not care so much about an individual estimate, but…

In general, I do not give much on a single estimate, because errors in precision & accuracy outbalance each other frequently.

What I do care of is:

Tracking and book keeping on scope items, their business value, estimated over the complete lifecycle from ideation until operations – including bugs, risks and impediments (and I like to use the agile way of capturing it in a well structured backlog)
Risks on not having clear requirements on the scope item to be estimated
Making estimations on scope items in the context of other scope items
The accuracy at the set of estimates I am getting
Having the right team doing the estimates and
Having a “Product Owner” that is able to answer, detail and discuss requirements with the estimation team.

…I care about the estimation process

The process of estimation starts with a clear understanding of roles, responsibilities and obligations related, as well as a well structured, understood process and resulting follow up activities.

I see three roles involved:

Scope owner or Product owner
Facilitator of the meeting
Estimating party

Usually several scope items should be estimated in a session to make best use of time.

First things first: Norming the Story Point scale

If you run the first estimation session in the context of a project or a team, you need to scale the Story Point metric to be meaningful.
For that you define two reference stories, a smaller one to be attached to a number at the lower range 0.5 to 2 Story points and one to a value in the lower mid range from 8-20 max.

This allows to compare all other stories to those reference stories.

This is run by the facilitator who picks initial stories and the team to add story point values to them – or recommend to use other stories if needed.

Estimation as team work

Complexity via Estimation Poker

For all estimations I suggest a team to estimate a couple of scope items in a session of 1 to 2h via Estimation Poker.
It needs the Product Owner to present and explain the scope item related requirements, context and acceptance criteria.
The team can ask and clarify their understanding – or reject the scope item. This can happen in case the product owner provides only vague ideas and lack of clarity on a scope item.

Poker means that all team members consider complexity individually and on 1 – 2 – 3 raise cards or fingers or call out their individual estimates at the same second.

This allows the facilitator and all participants to understand whether the variance between estimates is narrow or very broad. The broader the individual estimates are, the more likely is lack of clarity or aspects judged differently. In this case, asking the team members on assumptions and rational behind their estimate reduces risk of misunderstanding and should lead into a next estimate on the same subject with a narrow variance.

Complexity vs. Time / Effort

If you are in a running project, you can track progress of what leads to a factor between team progress, teams work capacity and complexity of scope items delivered per period (sprint).

Team velocity is how that factor called.

But If you are budget estimation on a project – eg. to answer a request of proposal etc.- a risk occurs: You have no evidence on velocity and need to do educated guess work on it.

The approach I am choosing frequently in those situations is:

Take the reference stories used to norm the Story Point scale prior to the complexity estimate.

Frame clearly what the estimate focusses on:
Only concentrated, focussed work is considered . Disturbance, meetings , overhead admin has to be ignored.
Then ask for the amount of time would you need to implement.
Claim your expectation that work is only accepted and ready – if they are QAed, integrated and can be brought into production just by a “click”.

Start with the referenced scope items – so that it can be brought into production just by a “click”.

Then I get a factor. I put that factor in the spreadsheet / backlog keeping all estimated scope items and add a “time needed” column besides the complexity column and use the factor to transfer all complexity into “ideal man hours” how I call it.

Then the team does a sanity check: Complexity vs. Ideal Man Hours. The factor or individual complexity parameters are matter of adjustments – what we do time boxed – to get a good feeling on the portfolio of scope items.

From Estimates to Budgeting

This is raw data, the program/project owner needs to take and add buffers. It is also important to know who owns what buffers and factors!
Usually my experience is, that a software development team is productive for max. half of a working day on the scope items. The other time they are socialising, doing admin stuff, attend meetings etc. – in a productive and supportive environment.

So a of 2 is needed to come to ideal man hours to actual hours in a on prem, well aligned and professional team.

If the team is distributed for more than 1 day a week, I add 10 percent for overhead.
If the team is in an offshoring context , I add 10 – 15% of additional overhead.

Usually I add 30% of risk buffer for time and material, and 50% in case of fix price.

That builds the estimate on what it takes to run the project. And buffers etc. are also the risk that I as a program lead and the team will take.
What we estimate is also our commitment to stakeholders.

Stakeholder claim efforts are too high

Or: Internal negotiations are the hardest

In case stake holders claim that your estimates are too high – damn – this is not a problem of your estimate. Your responsibility is to deliver an accurate, realistic perspective on efforts to help making decisions – and delivering an estimate and effort budget is already a commitment from you and the team!

Stakeholders have their own interests, and levers to adjust accordingly. They need to make own choices and need to make decisions and their own commitments:

Reduction of scope
Sponsoring budget for free – to win a strategic project
Blow the whole thing off – and… we are used to be cynical in IT, aren’t we
Find someone else

Final Comments

I hope this is helpful to you. For me the estimation and budgeting process is key to make reasonable projects in IT, because

It is a chance to review and block insufficient scoping, context and hidden risks
It is creating expectations by all stakeholders involved
Everyone has a chance to give a clear commitment on the project – or to refuse and wave a warning flag
It is the base for all future negotiations on change requests etc.

And hopefully reduces friction and conflicts during delivery as much as possible.

There are many other resources and I suggest to look for Backlog Grooming, Henrik Kniberg’s great video “PO in a nutshell”, Scrum or Kanban related resources etc.