Every March, developers worldwide each try to make a roguelike game in less than 168 hours as part of the 7DRL Challenge. Roguelike games are hard, involving procedural level generation, pathfinding, line-of-sight algorithms, and more dijkstra maps than you can imagine.

Most people don't survive the week. 0

Years of 7DRL challenges has taught me that development velocity is the most important thing to optimize for. It doesn't matter how perfect your code is if it doesn't make it into the hands of the players in time.

A few weeks ago I wrote the first part of our little expedition, which explored the various memory safety approaches. Today, let's talk about how they might help or harm development velocity!

You might learn something new, like:

• Modern non-memory-safe languages can use recent techniques to zero in on memory errors so quickly, that they can be fast to develop in.
• Reference counting has a hidden superpower that can help track down logic problems.
• Borrow checking can sometimes help and sometimes hurt development velocity, even after the initial learning curve.

Why it's important

Developer velocity is important. Not just for delivering a complete game in 7 days, but in a lot of every day software engineering situations too:

• A startup needs to get a working product into the hands of its users before the investment money dries up.
• One needs developer velocity to be able to respond to unforeseeable events, such as market opportunities or new regulatory requirements. 1
• A company needs to reduce costs to stay in business, including reducing development costs.

Some napkin math to illustrate that last one:

• Google used 15.5 terawatt hours of electricity in 2020, most which went to data centers. We'll conservatively assume rather expensive electricity ($0.199/kwh for CA). That comes out to $3.085 billion.
• Google has 27,169 software engineers. Some are higher, but let's conservatively use the entry level average yearly compensation which is currently $178,751. That comes out to $4.856 billion. 2

As you can see, software development can be much more expensive than power usage, so it makes sense to primarily optimize for development velocity. 3

The Language Factor

The choice of language, and its memory safety approaches, is a big factor in development velocity.

There are generally four approaches to memory safety:

• Manual memory management (MMM), like in C, Ada, Zig, Odin, etc.
• Borrow checking, like in Rust, Cone, Cyclone, etc.
• Garbage collection (GC), like in Java, Go, Python, Javascript, etc. 4
• Reference counting (RC), like in Swift, Nim, Lobster, etc.

There's also a fifth approach, generational references with regions which we'll talk about elsewhere; this series is comparing the more traditional approaches.

But by the end of this, you'll have a much better idea of which is best for a particular situation, whether it be a game development challenge, web server, or anything else.

What Actually Is Developer Velocity?

Development velocity is a nebulous concept, but I would say it's how fast we can expand, modify, and maintain our codebase to deliver value to the user and not cause too much collateral damage.

The "deliver value to the user" is very important here. It means doing something that will help the user in a specific way. Without that clear goal, we can get caught up in making our features absolutely perfect according to some arbitrary artistic criteria. As anyone who has launched a product can tell you, it's better to have two solid, tested, flexible features instead of one absolutely perfect one.

The "too much" is also important. There is often a tradeoff between moving fast and keeping things absolutely correct. A big part of software engineering is weighing the value of new features against the risk and relative severity of any bugs that might slip into production for a while.

With that in mind, let's see how the various approaches do!

Manual Memory Management (MMM) can be slow

People that come to C from higher-level languages often think that they should code like Java but manually inserting malloc and free calls. Let's call this style "naive C".

One can become pretty skilled in mentally tracking where these calls should go, but the approach tends to fall apart after a while: it isn't resilient to changes and refactoring. It's common to accidentally change the code and break some previous assumptions, resulting in a memory bug.

Memory bugs are notoriously difficult to solve. Naive C doesn't have great development velocity.

It can also be fast

...and that's why experienced C developers don't really use this "naive C" style of coding.

Instead, they:

• Use safer practices which drastically reduce memory problems, such as Architected MMM where we use arrays and arenas instead of malloc and free, always copying in/out of tagged unions instead of pointing into them, and so on.
• Use static analysis to enforce safer practices, such as SPARK.
• Use things like Address Sanitizer and Valgrind to detect buffer overflows, dangling pointers, memory leaks, and double-frees.
• Use special allocators that don't reuse memory, to better detect problems.
• If using C++, using RAII to automatically prevent double-free and memory leaks at compile-time.

With these, developer velocity can be stellar even with an MMM language. Modern MMM languages even include these kinds of mechanisms, such as Zig's Release-Safe mode.

There is a slight cost to these memory approaches. Address Sanitizer increases run-time by 73%. This makes it almost as slow as garbage collection. 5

However, that doesn't matter if we can just enable it in debug mode to get the error detection improvements for developer velocity and turn them off in release mode, just like assertions. As we talked about in Part 1, we wouldn't want this for a public-facing server, but it's a great tradeoff for many games and settings like webapps and mobile apps where any security issues can be sandboxed away. 6

Google Earth proved that this strategy can work well. In an average Google Earth quarter, only 3-5% of bug reports were traceable to memory safety problems, and Address Sanitizer made them trivial to reproduce in development mode.

Another developer velocity benefit from MMM languages is that they aim to be as simple as possible to keep compile times low. Languages like C, Zig, and Odin tend to compile much faster than more complex languages like Scala and Rust. This greatly helps developer velocity. 7 On top of that, Zig's simplicity helps its ability to hot-reload code changes which could make it one of the fastest native languages to develop in.