Agile software development methods create speed and flexibility in software projects – but why? The answer may surprise you. Agile methods work because they reduce feedback loops dramatically. Modern agile methods also improve communication and (to some extent) development methods, but most of the improvement in speed and flexibility is a result of how the flow of work is managed.
Working in Small Batches Shortens Feedback Loops
Think of your development process as a system or machine for turning ideas into saleable stuff. In traditional methods, all the “stuff” moves together in one enormous batch through all the activities in your development process. No feature can be coded until all have passed design freeze. No feature can go to testing until all have been coded. Because the batch size is so large, feedback is very slow. Bugs in a feature are not found until months after it has been coded, you can’t find out whether customers like the features until acceptance test, and you don’t know how long it takes to finish an average feature because you haven’t finished anything.
In agile methods, the same “stuff” is split into small batches of features, where each batch delivers value independently. Each batch proceeds through all the development activities before starting another batch. Suddenly a feedback loop of many months has been cut to a few weeks. Bugs are found while the design is still fresh in the developer’s mind. It’s possible to show users what you have and find out if it’s what they really want. Even better, it may be possible to release a batch and collect payment as soon as that batch has completed. Value is delivered faster and with less effort.
Small Independent Batches Also Create Flexibility
In addition, the project is more flexible because each batch is independent of the others and delivers value on its own. This makes it easy to change the order of batches, drop batches which haven’t yet been started, and add in new batches.
In short, one of the main reasons agile works is because the work has been split into small, independent batches and each batch is finished before another batch is started.
Why Queueing Theory Matters
This may seem like magic, but it’s not. There’s a science behind it, known as queueing theory. Queueing theory is a mathematical study of batches moving through systems of activity. Using queueing theory, it’s pretty easy to demonstrate why the batch size makes so much difference.
The principles of queueing theory not only explain why agile works, they can tell us a lot about how to fine-tune agile software development – and the same principles can tell us how to get the same speed and flexibility in other types of work.
For instance, a project will not necessarily get faster if work is split into any old type of batch. This has been demonstrated in software quite a few times – the batches have to deliver value to the end customer, and they have to finish. You will not get agile results by splitting design documentation into batches, nor by ending your sprint with delivery into system test rather than deployment.
Queueing theory tells us what attributes the batches must have. And queueing theory explains how to manage different types of batches – batches which are evenly sized vs. batches which can be different size. Batches which have interdependencies vs. batches which don’t.
Want to learn more about queueing theory? Start with What is Queueing Theory and Why Should I Care?
Or, read more about the history of agile from a queueing theory perspective in The Agile Manifesto Today
First version published November 2020.