July - 7th agile principle

2019-07-20

Working software is the primary measure of progress.

We are getting to one of my favourites. The seventh agile principle states that working software is the primary measure of success. This statement is so plain that First Officer Spock might raise an eyebrow and say: “Quite obviously so.” Indeed, the idea is simple. Working software directly translates into business value and the ultimate goal of any software project is to provide functional, usable, and reliable software that allows users to get their task done better and faster. But how do you measure it? Perhaps it is not all that obvious. To illustrate the case, let’s consider an example from construction.

Let’s say we are building a house. If we apply iterative methods to house construction, we would start with some room and finish it, perhaps the bathroom. So we got a working bathroom, tiles, taps, sinks, plumbing and everything included. Next we add a first bedroom. Now the owners can sleep in it and use the bathroom. We have provided value to the stakeholders. That’s very agile of us. But what about the roof? Or the foundation? Can we add it later? Evidently, the bedroom and bathroom require some sort of foundation. We already built one, but it’s not going to support the entire house. So we will have to change the foundation as we add rooms. And the roof will have to grow as well. It is easy to see that this approach doesn’t really work for construction. As we “refactor” the building, we would have to tear down and rebuild structures often, which would be extremely impractical and prohibitively expensive.

meeting room

Construction is therefore done in a different way. For instance, we start with excavation and cast a concrete slab foundation. Then we would add bearing walls, columns, a roof truss, pipes, cable conducts, windows, doors and railings. This is followed by tiling the roof, adding plumbing and electrical lines, connecting the house to the grid, putting in the floors, painting, adding finishings and decorations. House construction proceeds in well-defined phases and it does not provide any value to its owners until the last phase is completed and the house is ready to move in. Actually, almost every physical artefact is built that way. Bridges, tunnels, cars, planes, and machines are not built iteratively, but they proceed through defined phases from the drawing board to the final product.

Because everything made of solid matter is built in that manner, it is natural that people started building software in the same way. Traditional project management and the waterfall model are tied to a process that mimics the physical world. You begin with a complete set of specifications detailing the final product in every aspect. Then you implement the software module by module. For example, there may be a database module, a user interface module, a business logic module, a messaging module and what not. Then you bring all these modules together and code the instructions that perform the defined functions. Then the software goes through a testing stage and corrections are made where necessary. After that, the product is ready for delivery.

So, how do you measure progress in such a scenario? It is not entirely obvious, because it depends on specifications and procedures. Is a house 50% finished once the shell and the roof are completed? Depends on how intricate the fittings and finishings are going to be. If you put in stucco work and tessellated floors, it could be a long way. Traditional project managers use Gantt charts to measure progress. These charts model schedules and phase interdependencies. Because schedules often rely on assumptions and interdependencies are sometimes overlooked, these charts are not super accurate. Other traditional tools include task lists, bill of materials, and assembly plans which are often represented in complex spreadsheets.

These were also the typical tools of trade in software development in the 1990s. If you asked a software project manager back then about the current state of progress, you might have gotten a complicated answer. For example: “The database module is almost complete and 90% of the SQL is programmed. The UI module is finished. We are now working on the data transformations and the storage engine. This will take about two month until integration.” These words may have been accompanied by some diagrams. You would be in awe of the professional competence of this manager and full of optimism about the early conclusion of the project. But let’s be honest, you would not have any tangible metrics for actual progress and therefore no real clue about how the project is going.

This was the past, however. The agile development process is fundamentally different. Because the product is not physical, it’s more like building a house room by room. Ongoing code refactoring ensures that we do not end up with a smoregasbord, but with a proper house. In this scenario, measuring progress becomes actually quite trivial. You can tell by the number of rooms which are ready and usable. In other words, to lay the construction metaphor at rest, we construct software feature by feature, not by modules. We can measure progress by the number of features completed. Features can be broken down into user stories and user stories can be broken down into concrete units of work. So, at this level, estimating and measuring progress is viable and uncomplicated. Yet, the traditionally trained project manager may stare us in the eye and object: “How can you measure progress at all if there isn’t an overarching plan and the total number of features is unknown?”

This is in fact an entirely different story. The absence of a finish line so to speak, implies that an agile team never makes promises about project completion. For example, you will never hear: “The project is 50% complete.” However, you might hear: “The Sprint goals are 50% complete.” The absence of vague long-term predictions is counterbalanced by an abundance of concrete short-term predictions. An experienced agile team is also likely to have a realistic idea about its velocity. For example, you might hear that the team has an average velocity of 60 story points per iteration which may come down to 6-8 new features per iteration. So you can quite easily tell how many features the team can handle within two months or within half a year. It turns out that this information is more useful for business stake holders than vague predictions about project completion.

To summarise, measuring progress in an agile team is done at a micro level. It is therefore quite precise. Measuring progress at a large scale is difficult in any setting, unless the process is repeatable, which in software development it’s mostly not. However, data from past performance can be used to make reasonable long-term predictions.