milen.me — Premature Optimization

Premature Optimization: Universally Misunderstood

Published on 28 Aug, 2023

“Premature Optimization”

You might have come across the famous software optimisation quote popularised by Donald Knuth:

Premature optimization is the root of all evil.

– Sir Tony Hoare

It has been commonly interpreted as “don’t think about performance in the beginning, you can fix any performance problem later”. This interpretation is completely and categorically wrong.

Original Quote

It’s very common for statements to lose their original meaning when context has been stripped and that’s exactly what happened here.

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

– Sir Tony Hoare

The short version is missing a crucial part: “small efficiencies”. At the time the quote was made, “small efficiencies” referred to techniques like minimising the number of instructions used.

With the additional context, the quote takes on a significantly different meaning: it’s making a statement only about micro-optimisations ("small efficiencies"), not about performance in general.

Hoare was simply advising against micro-optimisations without finding the hotspots first: the “premature” part refers to lacking measurements.

Hotspot Optimisation: “Make it Fast Later” Fallacy

It’s quite tempting to adopt a “ship now, make it fast later” approach. While optimisations will improve performance, it won’t change the fundamental performance envelope. As tech stack fundamentals, access patterns, data dependencies and data structures are baked into a design, it’s not possible to hotspot your way out of a slow architecture.

Daniel Lemire perfectly explains this in Hotspot performance engineering fails:

Developers often believe that software performance follows a Pareto distribution: 80% of the running time is spent in 20% of the code. Using this model, you can write most of your code without any care for performance and focus on the narrow pieces of code that are performance sensitive.

Sadly, it does not work.

Charles Cook encounters this with a configuration client which performs a lot DCOM calls:

Again, optimizing this code would require a lot of re-working: optimization after a design has been implemented nearly always involves much more work than incorporating it into the original design.

In Performance Excuses Debunked, Casey Muratori astutely notes that if optimising hotspots was the solution to serious performance problems, software wouldn’t have to get rewritten to make it faster:

If Facebook’s performance problems were concentrated into “hotspots”, why did they have to completely rewrite entire codebases? Why would they have to do a “ground up” rewrite of something if only a few hotspots were causing the problem? Why didn’t they just rewrite the hotspots? Why did they have to rewrite an entire compiler in a new language, instead of just rewriting the hotspots in that language? Why did they have to make their own compiler to speed up PHP and Hack, instead of just identifying the hotspots in those codebases and rewriting them in C for performance?

Designing for Performance

If you want to have fast software, you must think about performance from day one. This includes thinking about the tech stack, the architecture, the data access patterns, the data dependencies, the networking and how it all fits together.

Charles Cook summarises it quite well:

Its usually not worth spending a lot of time micro-optimizing code before its obvious where the performance bottlenecks are. But, conversely, when designing software at a system level, performance issues should always be considered from the beginning.

A good software developer will do this automatically, having developed a feel for where performance issues will cause problems. An inexperienced developer will not bother, misguidedly believing that a bit of fine tuning at a later stage will fix any problems.

A Revised Quote

When having conversations about performance in the future, I’ll be using a revised version:

Premature micro-optimization is the root of all evil.

And that one, I complete agree with.

References

← Back to Writings

Copyright © Milen Dzhumerov
Email —

RSS —

Mastodon — Twitter