The Problem with "Overengineering"

07 Mar 2020

I worry about using terms which have heavily overloaded meanings. In software discourse, “simple” and “complex” may be the most pernicious, and using them without referencing a specific definition is virtually guaranteed to cause misunderstanding and strife. In the same vein, I’ve recently developed an aversion to the term “overengineering”. It can equally be applied to adding excessive layers to a system, writing extensible code, or just thinking hard about code before you write it, and these are all very different things. In this post I’m going to dig into a few definitions of the term, and hopefully show why I object to each of them. By the end of it I hope you’ll be convinced that “overengineering” is just an awkward term for “bad engineering”, and that the way to avoid it is to do engineering: more and better.

A few definitions #

First, let’s try to define the adjective form:

A software component is “overengineered” if it contains excessive layers of abstraction or indirection, which make it cumbersome to use. The term is most often used when these layers are intended to increase the component’s generality, in order to support perceived future use-cases.

This is hopefully unobjectionable; it fits with the usages I’ve seen in practice, and has room for some subjectivity without being totally meaningless. Now let’s try a couple of potential definitions for the noun case:

“Overengineering” is the act of producing overengineered software.

Tautological! Both obviously true, and obviously useless. Maybe we should try a literal reading of the term?

“Overengineering” is the act of engineering too much.

If a person had not heard the term used before, this is likely the meaning they would assign to it. As a result, this is also how a portion of developers interpret it. Finally, here’s a possible definition that I find more accurate:

A person is “overengineering” if they are so focused on either the future possibilities surrounding a software component or a preconceived notion of the component’s optimal architecture that they produce a suboptimal solution, which causes problems both immediately and later on in the actual future.

The reason I object to the term “overengineering” is that these definitions wildly diverge: the solutions we call “overengineered” are not a result of engineering too much, but rather of engineering badly.

Tradeoffs #

My favorite definition of “engineering” is “the act of managing tradeoffs”.¹ Our job titles call us “software engineers” if our job is mainly programming, but this is superficial: Software engineering is a body of knowledge, a toolbox of techniques, and the discourse and practice surrounding these things.

When a developer produces a needlessly clunky component, whose awkward abstractions get in the way, did they successfully manage their solution’s tradeoffs? Did they successfully learn from the experience of software engineers as a whole?

Some extremely common engineering maxims warn against these misimplementations. YAGNI is perhaps the most common, exhorting a developer who considers adding a speculative feature that “you ain’t gonna need it.” Maslow’s hammer is a well-understood cognitive bias: “when all you have is a hammer, everything looks like a nail.” Remembering this provides a bulwark against being captivated with a certain preferred pattern, and can help a developer remember to focus on the reality of the problem at hand. Basic familiarity with these concepts shows that engineering knowledge can guide developers away from “overengineered” solutions.

What’s the harm? #

One might (justifiably) ask why all of this matters. We use clunky language all the time and mostly get on fine, so why is this any different? My reason for writing this post is that I find the term “overengineering” especially insidious, because it conflates the problem with the cure. The term blames a focus on engineering discipline for poor outcomes, but what engineering discipline is is the sum accumulated knowledge that our industry has on how to avoid poor outcomes.

When we denigrate the act of thinking hard about our code and weighing tradeoffs, we open pandora’s box. There are many, many common errors and anti-patterns beyond premature or excessive abstraction and indirection. Vaguely recommending that developers “avoid overengineering” might make some of them think twice before speculatively preparing for potential features, but is just as likely to be taken as advice against following good engineering practices. There are many engineering maxims which could seem to be discouraged by careless use of the term: “code to interfaces, not to implementations”, “separate the what from the how”, “write automated tests”, “choose composable interfaces”, etc. Programming without these guidelines in mind is a sure way to produce worse results, not better ones.

Quality in the present #

Preparation for the future doesn’t have to mean making things worse in the present. Oftentimes, a composition of lightweight abstractions is both more extensible and easier to understand. To avoid producing “overengineered” solutions, use your engineering skills and knowledge to build solutions which are good right now. If and when the time comes to change or expand requirements, you’ll have good code to work from. If this time never arrives, you won’t have built and dealt with the extra baggage unnecessarily.

The flip side of this is that building low-quality solutions isn’t justified by avoiding “overengineering”. Incurring technical debt by preparing for hypothetical future flexibility is an error; so, too, is incurring technical debt by relying on a future refactoring or replacement of the code being written now. If code is complex, concerns are coupled, and tests aren’t up to par, the system’s quality will degrade, and discarding or refactoring the component in the future becomes more difficult.

Current thoughts about the future are a terrible guide for software decisions in the present. This is true whether these thoughts are assumptions that increased functionality will be relevant, or whether they’re the assumption that decreased quality won’t.

My final argument in this piece is this: We should not only implement in the present, we should also evaluate in the present. Rather than say that a component is “overengineered” and buying into its conversation about the hypothetical future, focus on the present, and say how and why the component violates good software engineering practice in the here-and-now. Rather than caution developers against “overengineering”, encourage them to learn about software quality and think about how they can make their code clear, reliable, and composable. We should be careful that we don’t accidentally tell developers to do less engineering when they actually need to do more.

---