The Waterfall process and predictive planning

The traditional delivery model known as Waterfall was first shown diagrammatically by Dr Winston W. Royce when he captured what was happening in the industry in his paper, Managing the Development of Large Software Systems, Proceedings WesCon, IEEE CS Press,1970.

In it, he describes a gated process that moves in a linear sequence. Each step, such as requirements gathering, analysis or design, has to be completed before handover to the next step. 

It was presented visually in Royce's paper in the following way:

The term Waterfall was coined because of the observation; just like a real waterfall, once you've moved downstream, it's much harder to return upstream. This approach is also known as a gated approach because each phase has to be signed off before you can move onto the next. 

He further observed in his paper that to de-risk this approach, there should be more than one pass through, each iteration improving and building on what was learned in the previous pass through. In this way, you could deal with complexity and uncertainty.

For some reason, not many people in the industry got the memo though. They continued to work in a gated approach but, rather than making multiple passes, expected the project to be complete in just one cycle or iteration.

To control the project, a highly detailed plan would be created, which was used to predict when the various features would be delivered. The predictive nature of this plan was based entirely on the detailed estimates that were drawn up during the planning phase.

This led to multiple points of potential failure within the process, and usually with little time built into the schedule to recover. It felt almost de rigueur that at the end of the project some form of risk assessment would take place before finally deciding to launch with incomplete and inadequate features, often leaving everyone involved in the process stressed and disappointed. 

The waterfall process is a throwback to when software was built more like the way we'd engineer something. It's also been nicknamed faith-driven development because it doesn't deliver anything until the very end of the project. Its risk profile, therefore, looks similar to the following figure:

No wonder all those business folks were nervous. Often their only involvement was at the beginning of Software Development Life Cycle (SDLC) during the requirements phase and then right at the end, during the delivery phase. Talk about a big reveal.

The key point in understanding a plan-driven approach is that scope is often nailed down at the beginning. To then deliver to scope requires precise estimates to determine the budget and resourcing.

The estimation needed for that level of precision is complicated and time-consuming to complete. This leads to more paperwork, more debate, in fact, more of everything. As the process gets bigger, it takes on its own gravity, attracting more things to it that also need to be processed.

The result is a large chunk of work with a very detailed plan of delivery. However, as already discussed, large chunks of work have more uncertainty and more variability, therefore calling into question the ability to give a precision estimate in the first place.

And because so much effort was put into developing the plan, there becomes an irrational attachment to it. Instead of deviating from the plan when new information is uncovered, the project manager tries to control the variance by minimizing or deferring it.

Over time, and depending on the size of the project, this can result in a substantial deviation from reality by the time the software is delivered, as shown in the following diagram: