Semantic Versioning Will Not Save You

The widely used Python package cryptography changed their build system to use Rust for low-level code which caused an emotional GitHub thread. Enthusiasts of 32-bit hardware from the 1990s aside, there was a vocal faction that stipulated adherence to Semantic Versioning from the maintainers – claiming it would’ve prevented all grief. I will show you not only why this is wrong, but also how relying on Semantic Versioning hurts you.

Dedicated to Alex and Paul who are willing to take the heat for the rest of us.

Version Numbers

Let’s set the stage by laying down the ultimate task of version numbers: being able to tell which version of an entity is newer than another. This can apply to anything, but we’ll focus on software packages here.

The software community has settled on interpreting version numbers as tuples of integers, separated by periods, with a precedence from left to right. Therefore 2.0 is newer than 1.10.0, which is newer than 1.9.42. The Python community has PEP 440 to formalize that.

And that’s all there is as far as this article is concerned: version numbers are unique, orderable identifiers of software releases.

Semantic Versioning

Over the years, well-intentioned people experimented with adding meaning to those numbers. The arguably most popular take is Semantic Versioning (SemVer). You have MAJOR.MINOR.MICRO and the promise is that as long MAJOR doesn’t change (aka a major bump), nothing will break and you can update your dependencies without prejudice. Unless MAJOR is a zero, which means YOLO time for the maintainer: anything goes.

Hyrum’s Law

Let’s start with broken promises and of course there’s an xkcd about it:

Workflow

by xkcd

It channels one of the fundamental laws of software development:

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

— Hyrum Wright, Hyrum's Law

What this means in practice is: even if the maintainer is pure of heart, extremely diligent, and super conservative with what constitutes a breaking change1, it is impossible to predict the ways a change can affect your users.

You want to claim that version 3.2 is compatible with version 3.1 somehow, but how do you know that? You know the software basically “works” because of your unit tests, but surely you changed the tests between 3.1 and 3.2 if there were any intentional changes in behavior. How can you be sure that you didn’t remove or change any functions that someone might be calling?

There’s obviously some nuance to this claim. I’m writing this from the perspective of a Python and Go programmer. And Python packages – through Python’s dynamic nature – are much more likely to suffer from breakage caused by unintentional side-effects. On the other hand I’m currently dealing with the fallout of unintended incompatibilities between two minor releases of a C library.

In essence, relying on updates not breaking if the maintainer doesn’t intend it, means relying on software being bug-free.

This does not mean that SemVer is bad or worthless. Knowing the intentions of a maintainer can be valuable – especially when things break. Because that’s all SemVer is: a TL;DR of the changelog.

What it does mean though, is that you can’t rely on the semantic meaning of SemVer and you must treat every update as potentially breaking. If a bump of the micro version never broke your production app, you just have to wait a bit longer. You too shall be blessed eventually – I promise.

On the bright side, I’ve seen major bumps come and go without affecting me at all. A major bump can only tell you about the existence of an intentional breaking change – but nothing about the impact, because it lacks the granularity.

Taking Responsibility

But some things have to be done. It’s better to do them, than to live with the fear of them.

— ‌Joe Abercrombie, The Blade Itself

The only person who is responsible for the health of your application is you. Your customers aren’t going to be understanding if you tell them that they can’t access their data because some teenager on a different continent broke your workflow by not adhering to SemVer as strictly as you’d like them to.

Pinning the major version number does not avoid your breakage. The best you can hope for is a temporary postponement. Postponing problems is generally a horrible idea because most problems only get worse, the longer you neglect them.

In practice that means that you need to be pro-active, regardless of the version schemes of your dependencies:

1. Have tests with good test coverage.

2. Pin your dependencies and their transitive dependencies to their exact versions.

   Go’s modules, Rust’s Cargo, and JavaScript’s npm do that by default. In Python, I use pip-tools but a plain pip freeze is better than nothing. You must separate your requirements that say Flask from your pin files that say Flask==1.1.2, along with Flask’s dependencies, and ideally their hashes. Otherwise, every build is a lottery.

3. Regularly try to update your dependencies to their latest versions. There are tools that help you with that.

4. If your tests pass, pin the new versions. If they don’t:

   • Fix them, then pin & commit the new versions.

   • If a single version of a package is broken due to a mishap and the maintainer intends to fix it in the next release, block the specific version from being considered for updates (e.g. Flask!=1.1.2, but not Flask<1.1.2).

   • If a package has intentionally made significant backward-incompatible changes and incremented their major version, block that major version (e.g. Flask<2), but – unless you have a contract about long-term support of the old major version – start working on adapting to the new major version immediately. Or look for alternatives.

     This is the only acceptable situation to pin the major version and it’s strictly temporary.

5. GOTO 3

Depending on the amount of churn, you can use the same process for open source packages. Tools like Dependabot will help you.

And that’s it. This is what you have to do, to prevent third-party packages from breaking your project or even your business. Most of the people that were angry at the cryptography maintainers about breaking their builds didn’t properly execute step 2.

There is nothing a version scheme can do to make it easier. It can only help you determining whether the breakage is on purpose or not.

This is also true for ecosystems like Rust or Go, that have SemVer baked into their packaging toolchain2. And most of the unintended consequences that I’ll enumerate next apply to them just as well.

Unintended Consequence: ZeroVer

Maintainers often feel like there’s an obligation to do SemVer. Some of that is self-imposed by not knowing about alternatives or thinking that “it’s the way it’s done”. Some of that is the result of demands by entitled users. And while SemVer promises freedom (technically you can break compatibility with every release, as long as you increment your major version!), in reality it delivers additional pressure and work.

The observable result of that pressure is what’s tongue-in-cheek called 0-based Versioning.

I’ve mentioned that in SemVer the maintainer can do whatever they want, as long as the major version is zero. That leads to many maintainers sticking to their beloved 0ver forever.

The SemVer standard is clear on the fact that a package that is fit for production, should be a 1.0. Unfortunately it’s culturally frowned upon to increment the major version3. So people stick with 0ver and the version number means absolutely nothing while claiming it’s semantic.

Thus, a package that has a 0ver version and at the same time claims to be production-ready is a paradox.

I don’t say that to throw shade on projects. I say that to demonstrate that SemVer is not a good fit for most projects and adds to maintainer burnout.

Unintended Consequence: Lack of Security Updates

The reason so many people were angry at the cryptography maintainers is that they are convinced that if only cryptography adhered to SemVer, they could just pin on the major version and nothing would ever break.4

As I’ve shown above, the “not breaking part” is nothing but a false sense of security and wishful thinking.

The pinning part is even worse, though: Most open source projects don’t have the capacity to maintain multiple major branches.

Open Source Reality

by CommitStrip

Therefore the moment you pin the major version of a package, it usually means that you won’t get any updates whatsoever once the package bumps its major version. In the case of security-sensitive projects – like cryptography but also web frameworks and their dependencies – this has potentially catastrophic consequences.

Unlike npm, Python mainstream packaging tools have no concept of vulnerable versions. You need extra tools or services to achieve that. That means that if you pin the major versions of your dependencies, your application will eventually be full of CVEs and you’ll never learn about it5.

But Wait – It Gets Worse!

If you maintain a public package and pin the major version of a dependency of yours, you transitively do this to the applications of your users.

Imagine an application depends on the wonderful urllib3 and your package does too. Now if you pin urllib3 to <2, the user of your package doesn’t have it in their power to ever receive an update from urllib3 again, once urllib3 bumps its major version to 2 and beyond.6 They may not even realize how far back they are.

On the other hand, if a new major version of a package surprisingly breaks your package, they can always add a pin themselves (see step 4 above) until you fix your package. But there’s no practical way for them to remove your pin.

Some Python packaging tools have adopted npm’s major-version pinning (^) by default, despite the lack of npm’s security features. Make sure to unpin them by hand if possible.

Unintended Consequence: Version Conflicts

This is a related problem to the last one. In languages that only allow one version of a package to be installed, pinning the major version of another package that you don’t control without knowing that the version of the other package is actually broken, you can unnecessarily cause version conflicts to your users that they can’t fix themselves.

Version Conflict

At least your users will know what’s going on. But they won’t be happy about it and it may bring some negative vibes to your tropical vacation, financed by the millions you’ve made from maintaining a FOSS project.

Summary

If you’re a maintainer and you like SemVer as an extra service to your users: go wild! I’m not here to tell you how to spend your time. There is value to adding semantic meaning to versions.

I am however here to tell you that applying SemVer to your project is entirely optional and if it stresses you out, or you’re stuck in 0ver land forever (meaning it does stress you out, but you don’t notice or acknowledge it), consider some of the alternatives.

As a user, I hope I have shown you that relying on SemVer:

• …can’t prevent breakage. At best, it can postpone it. Which is worse.
• …leads to version conflicts that will make your users unhappy.
• …leads to security problems that will make your boss and your customers unhappy.
• …adds burden on maintainers that will make the maintainers unhappy.

There’s also plenty of high profile projects that look like SemVer but aren’t: Linux, Python, Django, glibc…it’s fine!

To free myself from algorithms and to be able to add more context to my content, I’ve started a low-volume, announcement-only newsletter called “Hynek Did Something”. The announcement/director’s commentary for this post is its first issue and you can read it here.

If you find that interesting, I hope you consider subscribing! You can also find an RSS feed on the newsletter’s homepage if you hate e-mail!

1. Like the setuptools maintainers that have reached version 53.1.0 as of writing this (March 2021). The corollary is that any project with a single-digit version number that doesn’t use the 0.x escape hatch is likely treating their versioning loosely enough to not be truly SemVer. ↩︎

2. Putting foo = 1.0 into cargo.toml only matches releases from the 1.0 series. Bumping major in Go means creating a new import path. Which is so painful that even Google is weaseling around with their own projects. ↩︎

3. Hauptversionserhöhungsangst! ↩︎

4. Funny enough, a change in the build system that doesn’t affect the public interface wouldn’t warrant a major bump in SemVer – particularly if it breaks platforms that were never supported by the authors – but let’s leave that aside. ↩︎

5. Also be aware that many smaller projects never file for CVEs and just silently fix security issues as they go. Thus even GitHub’s fancy new security alerts won’t help you. ↩︎

6. This is less of a problem in languages like Rust, Go, or JavaScript where more than one version of a package can be installed. But you still have the problem that your package is running with an insecure version of a dependency. ↩︎