The beauty of the counterintuitive

Agile and lean thinking in software are entering the early majority phase of adoption. This is as always a confusing period since the beginners greatly outnumber the more experienced practitioners. Misunderstandings are rife and a lot of people are struggling to transition to a new way of thinking about software development. It is against this background that I get very frustrated when someone tries to sell the core ideas in agile and lean thinking as being “intuitive”. Selling agile and lean as intuitive is wrong and dangerous on two levels. I’ll explore each of these levels separately.

Intuition is influenced by experience

The first and reasonably obvious (some would say intuitive) reason that one should not sell agile and lean as intuitive is that you will simply confuse your audience. The first time most people come in contact with lean and agile ideas their intuition will tell them that these ideas must be wrong. Agile and lean are counterintuitive for a beginner. Just think about how you reacted the first time someone explained the concept of lean production (“you mean I should not maximize use of the most expensive resource by having buffers?”), refactoring code (“if it works, don´t fix it”) or agile scope/cost/time management (“if you do it that way you will lose control”). Most people’s minds reel when they start grappling with these ideas. By telling them that all this is intuitive you are just insulting them and making them feel stupid. Much better then to acknowledge that these ideas are counterintuitive for the beginner.

The intrinsically counterintuitive

Now for the really interesting part. Many people will argue that the counterintuitive becomes intuitive once you are an experienced practitioner. You just have to unlearn a lot of old stuff first. This argument is dangerous precisely because it is almost correct! The crucial question is whether all ideas become intuitive once you internalize them. I firmly believe that there are ideas that are intrinsically counterintuitive. Two obvious examples of counterintuitive things are relativity theory and quantum mechanics. Some of the predictions that are derived from these theories are just too strange to be understood at an intuitive level.

The idea that our brain is a “tabula rasa” has been proven false by many cognitive studies. Our brains are hardwired in some respects and our perception of reality is anything but completely objective. Accepting that our brains are inherently fallible is for me one of the core breakthroughs in the modern understanding of humans. All good systems have to be built around the notion of humans being error prone. On a personal level this means that we should always we wary of our reasoning. Even if you are an expert in agile development you should never forget that your mind sometimes will fool you into “believing” in the wrong thing. Things that are intuitively “right” will sometimes prove to be wrong. I think that part of the beauty of being an intelligent being is the ability not to follow your intuition.

Just as I was finishing of this piece I stumbled across new research that in some ways takes my argument even further:

Clever fools: Why a high IQ doesn’t mean you’re smart

Here is a great talk by John Seddon about systems thinking. A lot of counterintuitive insights:

Cultural change is free

/Niklas

Posted on 2009.11.02 at 15:16

Comments are off for this post.