The other kind of software debt

Technical debt

Most people in software are familiar with Ward Cunningham´s metaphor of technical debt. The metaphor is so powerful and intuitive that even the Wikipedia page does not really define what technical debt is, only what it leads to. Ward himself loosely defines technical debt as the difference between your current understanding of a problem and the understanding built into the code. He sees software development as a journey where your understanding of a problem is continually evolving. If you fail to update the codebase to reflect changes in your understanding, your code will become harder and harder to work with.

I like Ward´s definition but it only covers one type of technical debt. Almost all systems of interest today are built upon multiple frameworks. These frameworks are also continually changing, reflecting the framework developers´ evolving understanding of their problem. A codebase will over time build up technical debt with the growing difference between the way these frameworks are meant to be used and the codebase´s way of using them. You might argue that this debt is voluntary since you do not have to upgrade your frameworks. In my experience this is very often a bad strategy but that is a different discussion. What is indisputable is that there are situations where the cost of not upgrading a framework would be prohibitive.

Competence debt

So, if you were able to keep technical debt low, can you be sure that the system will be easy to maintain? No. There is a second type of software debt that is just as important as technical debt. I call it Competence debt and define it as:

“The gap between what is in your codebase and how much of it you understand.”

Figure 1: In this example the red parts of the codebase not covered by any developer is the competence debt. In most cases there will be a perceived competence debt (the bus factor) in the areas that only one developer is competent in.

Just as technical debt inexorably grows with time unless you fight it, competence debt also grows with time. The biggest difference between the two types of debt is that while technical debt grows faster the more you change a codebase, competence debt grows faster if you stop changing it!

Competence debt is therefore a problem that is most acute in mature systems where active development has ended. I think this is the reason there is so much more focus on technical debt. Technical debt is more sexy.

Mitigating competence debt

I find it strange that when people discuss pair programming they argue in terms of whether it is i faster to develop in pairs. For me, the real value of pair programming is in reducing both technical debt and competence debt. By pairing, team members broaden the areas of the codebase they are familiar with and increase overlap. In a similar manner, the value of refactoring is not just the reduction of technical debt. Refactoring is a great way of reducing competence debt too. It is only when you can change a system that you truly understand it.

What really kills systems

Over the years I have worked on a number of replacement projects. Looking back I realize that the real motive for many of these replacements was a severe competence debt in the old system. People would claim that the old system was impossible to maintain when the real problem was that they did not understand how it worked. Yes, technical debt made things worse since the confusing code and lack of automated tests made it frustrating to understand the system. The impulse to rewrite typically comes when too few of the original developers are left and the business is unable to find new developers that are able or willing to learn.

Unfortunately, replacing a system just because the competence debt in the old one is too high is seldom a good idea. The whole project lacks meaning for the users since the new system will not give them any visible improvements. Increased maintainability is seldom at the top of users wish lists. The cost of replacing is often an order of magnitude higher than expected. Ironically, the process of replacing a system often forces developers to study the old system, thereby reducing competence debt in it…

Conclusion

People often talk about software development as if it is a process that has similarities to manufacturing. This is misguided. Software development does not follow a fixed set of steps. I find it preferable to think of software development as an aged ship where the crew is continually working to save it from sinking. Some forces such as employee turnover pull the ship down. Other forces such as refactoring help keep the boat floating. Only by keeping a constant focus on both technical and competence debt can we keep the boat floating in the long run.

Niklas Björnerstedt

Note
I have written more on this subject:

• Misunderstanding technical debt
• A deeper look at Competence debt

Posted on 2013.11.11 at 16:54

Comments are off for this post.