Overengineering can kill your product

Overengineering can kill your product

BY Simón Muñoz ON NOVEMBER 16, 2021

Today’s post is not directed only to product managers. Founders, investors, or any other profile with enough skin in the game on any digital product or service could also take advantage of it.

I believe it because we will talk about one of the most prevalent issues when creating products: overengineering them. In my opinion, overengineering has killed more products than the absence of good development practices.

Before getting into detail, let me talk to you a bit about my background. Before being a product manager, I was an engineer. In fact, my formal training was in computer science. While during my career, I’ve always been closer to the business than to coding myself, I’ve created, scaled, and managed both engineering and product teams in equal measure.

So it’s not that I talk about overengineering from the outside. I’m guilty of having caused and suffered it. Because of that, I know it’s crucial to learn what it is, its costs, and how we can prevent it.

What is overengineering?

If we go to the strict definition of Wikipedia, over-engineering refers to the fact of designing a product in a more complex way than necessary:

Overengineering (or over-engineering, or over-kill) is the act of designing a product or providing a solution to a problem in an overly complicated manner, where a simpler solution can be demonstrated to exist with the same efficiency and effectiveness as that of the original design.

In the context of software, I like this definition from Paweł Głogowski:

Code or design that solves problems you don’t have.

Now you will think, who designs or codes to solve a problem that she does not have? It seems ridiculous, right? Well, hold on to the chair because, after two decades of career and having done it myself, I can assure you that over-engineering is not the exception; it is the norm.

Read this article on how to get design agencies and engineering teams to work together.

Overengineering causes

Nobody does it with bad intentions. On many occasions, overengineering happens because we try to anticipate the future and be ready for the unknown.

When we code a feature, it is easy to think that we can make it future-proof by investing a little more time “just in case”.

The reality is that nine times out of ten, that “just in case” never comes. But along the way, we have lost valuable time and increased the complexity of the project, something that we will carry throughout its whole life.

The general problem — (Image: XKDC)

Another cause of overengineering is often a lack of experience. In general, the more senior you are, the less prone you are to overengineer because you’ve already lived through quite a few situations where artificial complexity has exploded in your face.

The learning curve with respect to the experience of an engineer usually follows a pattern very similar to this:

1. You start by programming in a straightforward way.
2. You discover a paradigm such as object-oriented programming and jump on the bandwagon.
3. You read about design patterns and start looking for the opportunity to implement them in every situation.
4. After a few years of having suffered unnecessary complexity, you go back to writing straightforward code.

Code Complexity vs. Experience — (Image – flaviocopes)

Loosely defined requirements add to this problem as well. If an engineer does not have a well-defined problem, he will tend to overengineer to protect himself from uncertainty.

Boredom can also lead to overengineering. If an engineer does not have exciting challenges to face, he may end up complicating any problem simply by trying something new.

Overengineering consequences

When I said at the beginning of the article that overengineering kills startups, I was not kidding. It has two particularly perverse effects on any system.

On the one hand, it increases our development costs. If our engineers do not choose the simplest solution to address a problem, our costs in time and money increase, preventing us from iterating faster. Watch this MTP Hamburg talk on Iterating Your Voice Product, by Lisa Vigar.

On the other hand, it also increases your maintenance costs. Simple code is much easier to program, test, and modify. When we complicate it, the complexity can grow exponentially, impacting our iteration speed.

So I reaffirm myself. Overengineering kills products. Much more than the absence of good engineering practices. It is this way because to benefit from good practices you need to have product-market fit, while overengineering can prevent you from getting it in the first place.

Overengineering examples

The first that comes to mind is microservices-based architectures. They came like a wave a few years ago, and they should have taken down more projects than those they have consolidated.

I put them as an example of overengineering because they are not necessary in 99% of cases, especially for a startup that has to find market-fit and will benefit significantly from using a more straightforward architectural pattern like a “majestic” monolith.

If you succeed in finding market-fit and it turns out that you end up needing to switch to microservices due to scale problems, oh boy, that’s a good problem to have.

Premature optimization is often another typical example of overengineering. A common situation is preparing a system to absorb a large amount of traffic with an overly complicated infrastructure setup when you still don’t have users. In most cases, having a monolith running on a single server is all you will need to validate your business model.

One of the worst examples of premature optimization occurs when we spend a lot of time designing a system trying to prevent repeating ourselves in the future and having to throw away part of the work done.

It doesn’t matter how perfect your design or implementation is if you never see it working because you go broke before. The worst code on the planet that helps you validate a hypothesis is better than standing still for fear of not repeating yourself.

Related to the above, software rewrites are an obvious example of overengineering. Usually, engineers do not like to work on legacy codebases. Their natural tendency is to do everything from scratch. But as Joel Spolski wrote more than 20 years ago in Things you should never do, rewrites rarely serve their purpose and can even take your business away.

It’s obvious to say it, but your client doesn’t care how elegant your system is on the inside. Your client cares that you help him solve his problem. Every minute invested in not giving them value is a wasted minute.

How to prevent over-engineering?

The best way to prevent over-engineering from my point of view is to turn your engineers into true product engineers.

We can achieve it by involving them in the day-to-day business, explaining the why after each initiative, and linking it with the metrics that matter for the organization and its vision. Watch this MTP panel to find out more about defining metrics that matter.

We need to bring them closer to our users, inviting them to interviews and discovery sessions with them. You want your team to empathize intimately with your user’s problems, so they can quickly discard any engineering effort that won’t solve them as efficiently as possible.

Don’t expect your engineering team to be motivated to avoid complexity if you treat them as mere production chain resources whose sole task is to implement user stories from a backlog. They need to understand the why behind each decision.

Related to the above, define the problem well to reduce ambiguity. Your engineers need to know the why, but they also need to know what to expect. The more you can narrow down the problem, the less reason they will have to protect themselves from overengineering a solution. A good way to define the expectations of a system is by using service objectives using SLIs and SLOs.

Engineers are one of the most creative profiles in any company. If your team trusts your criteria, day-to-day ideas or initiatives will likely arise on their part that may show that they are thinking about a future “what if” scenario. When you have the intuition this could be the case, ask: How does this help solve our current user’s problems? What happens if we don’t do it now? The answers will help you to discriminate if it is a possible case of overengineering or not.

Lastly, more oriented towards founders, prioritize hiring senior engineers who have already had a few experiences in product companies. Look for war scars during interviews. Ask about their most painful experiences and how they dealt with them. Stick with those who put the user and simplicity ahead of simply technology solutions.

Mental models to prevent overengineering

YAGNI

The problem of overengineering is so prevalent in the industry that the engineers themselves have a term to refer to the situation of adding code “just in case”: YAGNI, an acronym for “You are not going to need it”.

YAGNI tries to prevent you from adding anything that is not strictly necessary to solve the problem you have in front of you because the reality is that most likely, “you won’t need it”.

The term KISS, “Keep it simple stupid”, refers to the fact that simple systems are easier to repair, evolve and maintain. Therefore, simplicity should be one of the goals of any design, avoiding any unnecessary complexity.

Worse is better

With Worse is better, we emphasize that there comes the point when having fewer options is preferable to having more. It’s this way because it can simplify the use of our product, making it attractive to a broader segment of the market.

In other words, it encourages us to maintain the minimally essential functionalities of a product, avoiding adding fat that can add complexity to it.

Conclusion

To wrap up, overengineering has the potential to destroy your startup, it can:

• Add unnecessary complexity.
• Increase development and maintenance costs.
• Reduce your iteration speed.
• Avoid you from getting market-fit.

Unfortunately, overengineering is no exception; it is the norm. For this reason, it is vital to know what it consists of and try to prevent it mainly by involving and bringing your engineers closer to your customers’ real problems.

Every minute we invest in development that doesn’t solve an actual customer problem is a minute wasted. Avoid falling into the “just in case” trap.

The graveyard is filled with exquisitely designed startups and products to scale to millions of users who never got the slightest bit of traction. Don’t become one of them.

Want to hear more from Simón? Have a read of his recent articles: