Before you commission a painter to decorate your home or a mechanic to fix your car, you get an estimate from them, right? You need to know how much it’s likely to cost and how long it might take. It’s just common sense.
What does experience tell us, however? How close are those original estimates to the final bill? It’s all too likely that the painter will find loose plaster that needs removing and the wall needs rendering and re-plastering. The mechanic is sure to find additional work required to get your car roadworthy again. In a 1951 cartoon for the New Yorker magazine, Syd Hoff drew a mechanic saying to his customer, “of course, that’s only an estimate; the actual cost will be more”.
If the painter or mechanic tells us in enough time, we can choose not to take on the extra work … and yet, all too often, we feel we have to fix these additional things. Who wants to live in a house with a potentially damp wall or drive in a car with potentially faulty steering?
How do we overcome this? A common reaction is to insist on a full and final fixed-price estimate, or ‘quote’ as it’s commonly called; so they work harder and longer to get more accurate estimates. Yet however hard they work, they still cannot really predict the unexpected.
Traditionally, project managers tend to focus on creating detailed estimates that can withstand scrutiny from the finance team. Of course, this is based on ‘known knowns’ with some contingency for the ‘known unknowns’ … and as Donald Rumsfeld is famously quoted “there are also unknown unknowns – there are things we do not know we don’t know”. Like the tradespeople above, we can never really predict the unexpected.
The more we invest in creating elaborate estimates, however, the more problems we cause. Detailed estimates can be seen as a binding quote, a target that distracts from delivering value, and focusing on delivering something—or even anything—to the agreed date and cost.
We beat ourselves up in post-implementation reviews, and tell ourselves to just try harder; Einstein, however, defined insanity as “doing the same thing over and over and expecting different results”; so there must be a better way.
Surely this isn’t true for agile projects? Don’t we just start with a high-level scope and work it out as we go along? Well, yes and no (or as they would say here in New Zealand, “yeah … nah”). While we don’t hamstring ourselves by creating detailed estimates, it is still vital that we get a feel for the size of the work we’re considering, and here’s why …
Forget about estimates being used to build beautifully crafted Gantt charts that force us to focus on tasks and not outcomes. There are three outcomes that need us to get a rough handle on how big something is.
It’s also interesting to note that estimating can be a really healthy activity, so long as the whole team collaborates on it together. It helps foster a buy-in from the whole team and ensure that everyone gets a common understanding of the scope and value to be delivered.
However the label of ‘estimates’ can be distracting.
To avoid setting an expectation that we’re talking about cost and time, when we estimate the complexity of a story, some of us like to refer to this as ‘sizing’ rather than estimating. In the 90’s when I first used Scrum and XP—before they were even called agile practices—we sized stories using t-shirt sizes (S, M, L, and XL).
Now, however, we use story points – a way of sizing stories relative to each other – so we find what will be the simplest story, size that as 1-point story, then compare another story to that, if it’s more complex than maybe it’s a 3-point story.
To make things more interesting, we don’t use a simple sequence like 1, 2, 3, 4, 5, etc. – instead we use a modified form of the Fibonacci sequence like 1, 2, 3, 5, 8, 13, etc. (as seen in ‘The Da Vinci Code’). This makes the jump between numbers larger the higher we go and drives us to choose which is smaller or larger.
While this is not an exact science, it is more than good enough for the three outcomes mentioned above, and as John Maynard Keynes said, “it is better to be roughly right, than precisely wrong”. This means, though, we still need to translate story points to rough time and rough cost.
Another practice central to agile projects is establishing the ‘definition of done’, that is to have a comprehensive understanding of everything required to say that a story is done and releasable, including items like user documentation, translation, advertising, etc. etc.
Provided we have a good definition of done, we can then take a couple of sample stories and calculate the effort required. From that we can get rough cost estimates for an investment decision, rough timing for release planning, and enough understanding to assist in story prioritisation.
Some, however, still find estimating by story points a distraction, and one response has been the debate around #NoEstimates.
If you have been following any of the major influencers on Twitter, you might have noticed some getting involved in discussions with the hashtag of #NoEstimates. While this sounds like a call to cease estimating altogether, it is really a call to stop sizing stories and instead just start developing.
This discussion arose, and has gained momentum, because the experience of those working on large projects has been that whether you size by story points or just count the number of stories, the tracking of throughput is about the same.
While this might partly be down to the more experienced delivery teams decomposing stories to consistently smaller sizes on a just-in-time basis—as this enables greater throughput of potentially shippable increments—the reporting looks similar even on projects still working with a range of very small to moderately large stories in each iteration.
While this makes story prioritisation simpler and enables the delivery team to get on with it quicker; our poor product owners and sponsors still need to know roughly how much it will cost and how long it will take. The good news is that we can still do this based on counting the number of stories, once we have sufficient metrics to make this feasible.
For any new team starting out on your agile journey, however, I would still strongly recommend you size with story points until you’ve reached the experience and maturity levels of these teams.
So to sum up, and to mangle a famous quote from General Dwight D. Eisenhower: “In preparing for [projects] I have always found that [estimates] are useless, but [estimating] is indispensable“; we still need to do it, whatever we call it and whatever we’re counting when we do it.
I am indebted to Esther Derby (estimates become targets), Mike Cohn (estimates for release planning and prioritisation), Ahmed Sidky (whole-team estimation), Martin Fowler (story points), Ian Mitchell (definition of done), Vasco Duarte (story count vs story points), Stephen Forte (metrics vs estimates), Neil Killick (experienced teams define smaller stories), amongst many others, for sharing their thinking on this topic – and I have linked to their respective articles in context above.
Note: this article first appeared in infoQ.