When the solution is the problem

How to prevent over-engineering & useless features in software

If all you have is a hammer, everything looks like a nail

Software engineers and experience designers believe that they can create and improve software by simply exercising their craft. Does the product suffer from low user engagement or service outages? “Let’s get into a room and whiteboard a solution,” they’ll say. There is a naive and widespread belief at work here. The belief is that design and engineering simply solve problems. This belief masks a wicked blind spot: every act of design and engineering, far from being a pat solution, is an “intervention in a complex situation” [Baumer & Silberman, 2011].

Building software is like surgery

Software is subtle and complex; myriad components interact in a fragile equilibrium of design, engineering, marketing and real world users. In its complexity and inter-relatedness, software is like the human body. And in spite of every effort to make the creation of software pristine and mathematical, it’s a lot more like blood and guts surgery. Risk, unforeseen consequences, and complications shadow every procedure.

Modern medicine dubs this risk as “iatrogenesis,” the unintentional harm that arises from medical treatment. As of today, iatrogenesis kills more people than any single cancer. In the United States, iatrogenesis kills between 3 and 10 times as many people as car accidents. Moreover, as unlikely as it may sound, only in recent times has medicine begun to save more lives than it takes.

In comparison to the discipline of medicine, software engineering is young and immature. It’s therefore no surprise that modern software is full of features and fixes that do more harm than good. Remember OpenSSL version 1.0.1, or the last OS update that bricked your smartphone? The cure is often worse than the disease. It’s time to acknowledge iatrogenesis in software. It’s time for software professionals to study and prevent the harmful effects of design and engineering.

There is no hammer

Professionals suffer from domain bias, or the Law of the Instrument. When faced with a problem, we tend to respond with the hammer that we have in our hands. We seldom question whether or not the problem would benefit from hammering. We seldom question whether or not we are faced with a nail at all. As a result of this failure to question, so-called “problem solving” is often “problem exacerbation.” To overcome the Law of the Instrument we must shift our mindset. The key to this shift comes from a concise gem of a paper, When the Implication Is Not to Design:

“Rather than imagining that technology design offers solutions to the problems of unsustainability, we suggest thinking of design as an intervention in a complex situation.” —Baumer & Silberman

How to stop over-engineering

Ask three questions

Now that we understand design and engineering as intervention in a complex situation, how do we prevent the unforeseen and unintended consequences of intervention? The answer is that we ask questions before we design a single pixel or write a single line of code. Baumer and Silberman offer the following three questions:

1. “Could the technology be replaced by an equally viable low-tech or non-technological approach to the situation?
2. Does a technological intervention result in more trouble or harm than the situation it’s meant to address?
3. Does a technology solve a computationally tractable transformation of a problem rather than the problem itself?” (i.e. is the technology a pale substitute for the real solution)

Shift the company culture

If you wish to prevent over-engineering and feature creep, the following cultural values are constructive:

1. Value not doing things. Start a conversation in your company about “less is more.” REWORK and Getting Real evangelize this message in a form that software teams will appreciate.
2. Consider extravention. That’s right, explore which features you can remove to improve the product. Antoine de Saint Exupéry captures extravention in one bright line: “perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away.” (If you’re looking for a more formal approach to extravention, read How to Cut Features and Enjoy it — 20+ questions to find the simplest design.)
3. Document the roads not taken. In order to overcome the Law of the Instrument and prevent revisiting the same ground in the future, document what you’re not doing and why you’re not doing it. Put the roads not taken on the wiki, even if your every instinct tells you to put them in a file drawer. Just as domain bias encourages us to start hammering without thinking, publication bias discourages us from documenting the things that we don’t do. (Publication bias is powerful enough to blind history itself. Can you name one person, living or dead, who is celebrated for not doing something?)

Keep creating

I do not advocate Luddism. Engineers and designers are at their best when they create on terra incognita. I do advocate a shift in perspective. Design and engineering are interventions in complex situations. To practice this is to practice humility and precision. Software, the people that use it, and the situations that create it are more complex than we take them to be. Expect surprising, nonlinear results from each intervention. To remember this is to prevent iatrogenesis in software. Primum non nocere, “first do no harm,” is a global maxim in medicine. It should also become the rallying cry of software professionals.

Further reading

Antifragile provides a model of complex systems that helps software creators to see around corners before they intervene.

“To sum up, anything in which there is naive interventionism, nay, even just intervention, will have iatrogenics.” —Nassim Taleb