Waterfall projects are more successful than agile projects. Wait! What?
For a university paper I am currently writing, I revisited the 2013 Chaos Manifesto. This report marked a watershed moment in the long history of the Standish Group and their biennial Chaos Reports that chart the factors that make projects successful.
What stood out for me, on this reading, was the diagram on page 25 that shows 49% of waterfall projects were successful, against 46% of agile projects.
I had to take a cartoon double-take.
How can this be true? Everything we have observed and learned indicates that agile projects are more successful. This seems counter-intuitive. It seemed like the clock was being set back by more than 20 years. If true, it could undermine everything I work for as an agile coach.
With some trepidation, I explored this a little more closely.
The Chaos Manifesto is a clarion call for working with small-scale projects (which it defines as less than one million US dollars); in fact, the whole report looks only at what makes small projects successful.
Within this context, looking at small projects only, apparently waterfall projects are marginally more successful. How can this be?
Is this perhaps because, if used at all, waterfall is best applied to situations where the problem and the solution are both clearly defined and unlikely to change? In Cynefin, this would be termed the ‘simple’ context which lends itself to smaller projects.
The success of agile practices over the last 20 years, and the recommendations from Standish in their Chaos Reports, have consistently raised the success factor of making projects smaller in scope.
As Standish put it, the “size of a project trumps methodology”; and this supports the realisation that even in the mid-90s, the lightweight frameworks then being espoused were an evolution of good practice, not a revolution dreamed up on a snowy mountain.
What Standish do say, though, is that using agile practices promotes working with smaller projects; with 45% of agile projects being small in contrast to just 14% of waterfall projects.
Beginning to feel a little more confident again, I referred to Standish’s other reports, as well as other sources, and was further reassured. The success rate on agile projects is between three to ten times more successful than waterfall.
In today’s business environments, especially commercial, the pace of change is ever-increasing, businesses are being faced with wicked problems that don’t have easy answers, and customers expect to be delighted and have short attention spans.
In these uncertain times, where the problem is not clearly understood or the solution is not fully defined, agile or lean approaches are still the most appropriate.
So it’s OK. While we still have much to learn and there is still new territory to explore, we are not facing the resurgence of waterfall practices. Phew!
