Welcome to issue #3 of Fresh Air, Deep Dive 🐳 — Jonah Group’s monthly newsletter, which highlights current topics of conversation in the software industry and how it may impact you.
For the April and May newsletters, we will be focusing on some of the unspoken and unrecognized assumptions about Agile, and challenging you to shift your thinking a little on these.
In this issue:
Agile’s Dirty Little Secret — It’s more expensive. Don’t tell anyone.
In the next issue:
Jonah Says "Stop Sprinting"— Your team is going to burn out!
Better than the MVP — What's missing from the “Minimum Viable Product?”
As always, we have some resources and suggestions for further reading.
“The Agile movement in software is part of a larger movement towards more humane and dynamic workplaces in the 21st century.”
— Rowan Bunning
If you’ve been around software teams, then you’re familiar with the term “Agile,” which is a well-established, and well-accepted set of tenets most teams use today for the delivery of modern systems, including most notably:
Iterative delivery (every 2-3 weeks)
Automated testing
Continuous integration and delivery (CI/CD)
Continuous feedback and reprioritization
The prevailing Agile methods of the day additionally use a variety of associated tools and techniques, like planning poker, issue boards, burndown charts, sprint retrospectives, velocity measurements, and leaders that help to remove obstacles. “Courage” to make changes is a central cultural tenet. All of this sounds great, and certainly not worthy of “YAAA!” (Yet Another Article on Agile).
We also know that before there was Agile, traditional “Waterfall” techniques were used to build software, in the same way that you might manufacturer a car. Each “phase” of delivery (Analyze, Design, Build, Test, Deploy) was gated by the prior one being complete. This had the distinct disadvantage that the process was brittle to change. Changes in a late phase caused rework of many decisions fleshed out in all prior phases, and long timelines may have meant that the world had morphed substantially by the time the water streamed over the rocks at the bottom of the cascade.
But as we all should know… things are never that black and white.
Methodology is more of a continuum than a discrete choice.
The dirty little secret of Agile is that the inevitable responses to regular feedback take time and effort, and cost money. And that means that the budget for the project must increase relative to a scope estimate that doesn’t incorporate feedback. It’s counterintuitive because the word “Agile” evokes feelings of speed, efficiency, and cost reduction. But being responsive to change isn’t free.
Does this mean that we should re-adopt Waterfall to save money on software projects? Comparing apples to apples, the answer is usually no: if the changes and reprioritizations integrated during an Agile project were to also be applied to a project run using a Waterfall method, the project overall would also likely become more expensive, even more so than one executed with Agile.
In practice, however, those changes are often not applied on Waterfall projects! At the end of a Waterfall, there is generally some grumbling about the fact that the team “got 90% of the way there but missed the mark on the other 10%.” But then a funny thing happens: stakeholders simply accept the way the system was built because the agreed-upon budget and schedule have been exhausted, and defer upgrades to a future phase, which may never happen. Everyone has champagne and pizza, speeches are delivered, and people go home relatively happy.
The rallying cry of the successfully completed Waterfall project.
While this doesn’t sound optimal from a “get what you want” perspective, in practice it does save the business some money. Not to mention, there is a class of change introduced in Agile projects simply because there was an opportunity to do so as opposed to because there was an overarching business need for them.
Waterfall projects also save the overhead of executing the iterative “machinery” every two weeks – e.g. spinning up analysis and design resources and efforts, conducting sprint retrospectives and reporting, and of course executing mini-UATs with the customer. For better or worse, there’s simply not as much of this happening on Waterfall projects.
So how do you navigate these tradeoffs? After you've completed your estimates, add 10% of the budget to allow for the cost of inevitable requests for continuous change. Only consider Waterfall if the project is short (<3 months), and if care has been taken up front to document functional and non-functional requirements in detail, to reduce multi-phase retrograde rework. If the project is short enough, what is delivered at the end of the project will approximate what is thought to be needed now. In the limit, of course, “short” devolves to the length of a sprint, and we’re back to Agile. We often find that a 3-week iteration cadence strikes a good balance between agility of deployment and iteration overhead.
On long (>6 month) projects, however, Waterfall should be considered off-limits - 6 months without user feedback is just too long. Everyone has a favourite panel from this famous cartoon, which exemplifies team role biases, and what happens when a software method doesn’t accommodate feedback well.
The folly of role biases on software projects.
One might imagine other dichotomies that need to be managed on the continuum between Agile and Waterfall. Waterfall is better at giving stakeholders cost certainty at the start of the project. Agile is better at reducing the risk of big-bang deployments and the horror of “integrate and test all at once.”
There are many of these, and as such there is “no one size fits all” software method. For longer projects, Jonah Group’s LightWave® methodology usually includes a Discovery phase, which aims to capture a middle ground between pure Agile and pure Waterfall. A practical approach tailored to the peculiarities of the project at hand is essential, here.
If you’d like to go further down the rabbit hole with these dichotomies, read our Mona Lisa article. Caution – due to its length there is a clear and present danger of being overtaken by sleep with this one, but if you persevere ‘til the end you'll gain a more subtle understanding of the forces at play.
