Development with Rust can be a lot of fun and the rich type system in combination with good tooling such as rust-analyzer can provide helpful hints and guidance continuously while writing code.

This, however, does not fully compensate for slower workflows that come from edit-compile-run cycles that require full program restarts plus the time it takes to recompile the entire program. Rust provides incremental compilation support that reduces the time for rebuilds often down to few seconds or for lightweight programs even less. But relying on, for example, macro-heavy code or types that make ample use of generics, compile times can go up quickly. This makes development with Rust quite different from lively environments and dynamic languages that might not just come with less build time but that might even provide feedback at the tip of a key.

Whether or not this impedes the ability to build software depends of course on what is built and on the approach a developer generally takes. For use cases such as games, UIs, or exploratory data analysis, quick feedback can certainly be an advantage — it might even allow you to make experiments that you would not have tried otherwise.

To enable such dynamic workflows when programming Rust, I have long wanted a tool that makes immediate feedback possible and hopefully easy to use. This blog post presents my take on the matter by introducing the hot-lib-reloader crate.

Table of Contents

Where this idea comes from

None of this is new of course. Lots of languages and tools were created to allow code changes at runtime. “Live programming” in Smalltalk and Lisps is an essential part of those languages and which is enabled by reflection APIs the languages possess as well as by tools that. Also Erlang’s hot code swapping plays an important role for the reliability of Erlang programs — upgrading them is not an “exception”, it is well supported out of the box.

“Static” languages have traditionally much less support for the runtime code reload. An important part of it is certainly that optimizations become much harder to achieve when assembly code needs to support being patched up later1. But even though modifying code arbitrarily is hard, dynamically loaded libraries are an old idea. Support for those is available in every (popular) operating system and it has been used to implement hot code reloading many times.

For C / C++, this mechanism has been made popular by the Handmade Hero series which presents this approach in detail (1, 2, 3). And also in Rust there have been posts and libraries around this idea (for example Hot Reloading Rust: Windows and Linux, An Example of Hot-Reloading in Rust, Live reloading for Rust, dynamic_reload).

Dynamic library (re-)loading is also what I am using in the approach presented here. The fundamental implementation is quite similar to the work linked above. Where this differs somewhat is in how you can interface with the reloader. By using a macro that can figure out the exposed library functions, it is possible to avoid a lot of boilerplate. In addition I will later show how this can be further customized to play nice in the context of frameworks such as Bevy.

“Hello World” example2

The simplest example will require a binary with some kind of main loop and a library that contains the code to be reloaded. Let’s assume the following project layout where the binary is the root package of a Cargo workspace and the library is a sub-package:

$ tree
.
├── Cargo.toml
└── src
│   └── main.rs
└── lib
    ├── Cargo.toml
    └── src
        └── lib.rs

The root Cargo.toml defines the workspace and the binary. It will depend on the hot-lib-reloader crate that takes care of watching the library and reloading it when it changes:

[workspace]
resolver = "2"
members = ["bin", "lib"]

[package]
name = "bin"
version = "0.1.0"
edition = "2021"

[dependencies]
hot-lib-reloader = "*"
lib = { path = "lib" }

The library should expose functions and state. It should have specify dylib as crate type, meaning it will produce a dynamics library file such as liblib.so (Linux), liblib.dylib (macOS), and lib.dll (Windows). The lib/Cargo.toml:

[package]
name = "lib"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["rlib", "dylib"]

lib/lib.rs is providing a single public and unmangled function do_stuff() (it could define state and other functions as well, do_stuff could also have parameters and return something):

// When using the source_files: ["path/to/lib.rs"] auto discovery
// reloadable functions need to be public and have the #[no_mangle] attribute
#[no_mangle]
pub fn do_stuff() {
    println!("doing stuff")
}

And finally src/main.rs where we define a lib loader that targets the lib subpackage. We use it in a loop where we check for library changes and then call the exported library function.

hot_lib_reloader::define_lib_reloader! {
    unsafe MyLibLoader {
        // Will look for "liblib.so" (Linux), "lib.dll" (Windows), ...
        lib_name: "lib",
        // Where to load the reloadable functions from,
        // relative to current file:
        source_files: ["../lib/src/lib.rs"]
    }
}

fn main() {
    let mut lib = MyLibLoader::new().expect("init lib loader");

    loop {
        // this reloads the lib should it have changed
        lib.update().expect("lib update");

        // This calls the reloadable function
        lib.do_stuff();

        std::thread::sleep(std::time::Duration::from_secs(1));
    }
}

Above is the part that matters: A new type MyLibLoader is created that provides a method MyLibLoader::do_stuff(&self). The method is automatically generated from lib::do_stuff(). Indeed, if we were to add a method lib::add_numbers(a: i32, b: i32) -> i32, a method MyLibLoader::add_numbers(&self, a: i32, b: i32) -> i32 would be generated. And so on.

To now run the app with hot-reloading enabled, start two cargo commands (this assumes you have cargo-watch installed):

Note that you will need Rust nightly. Either set it for this specific project with rustup override set nightly or run with cargo +nightly ....

• cargo watch -w lib -x 'build -p lib' which will recompile the lib when it changes
• cargo watch -w bin -x run which will start the app with hot-reloading enabled (and will also restart it when any file under bin/ changes).

Now modify the do_stuff() function, e.g. by changing the printed text or adding additional statements. You should see the output of the running application change immediately.

How it works

hot-lib-reloader uses the libloading crate which provides a unified and safe(r) interface around functions provided by the operating system to load and unload dynamic libraries (dlopen, dlclose, LoadLibraryEx, etc.). It watches the library file or files you specify using notify. When one of those changes it will mark the library as having a pending change. The next time you call LibReloader::update, libloading is used to close the previously loaded library instance and load the new version. Actual calls to library functions are then made by looking up a symbol with the matching function name, assuming it resolves to an actual function with the signature assumed, and then calling it. The define_lib_reloader macro is just syntax sugar around that.

For various reasons the reloader will not actually load the library file that Rust created but will create a copy and load that. Among other things this prevents from holding a file lock on Windows, allowing the library file to change.

In addition to that, the hot_lib_reloader::define_lib_reloader! macro takes care of some boilerplate. When declaring and using a reloader like

hot_lib_reloader::define_lib_reloader! {
    unsafe MyLibLoader {
        lib_name: "lib",
        source_files: ["path/to/lib.rs"]
    }
}
// ...
let lib = MyLibLoader::new().expect("init lib loader");
let result = lib.do_stuff();

the desugared version looks something like this:

let mut lib = hot_lib_reloader::LibReloader::new("target/debug", "lib").expect("init");
// ...
let result = unsafe {
    let sym = lib
        .get_symbol::<fn()>(b"do_stuff\0")
        .expect("get function symbol");
    sym()
};

In particular, without the macro it is necessary to keep the function signatures between where they are declared and where they are used in sync. To avoid that boilerplate, the macro can automatically implement the correct interface by looking at the exported function in the specified source files.

Caveats and Asterisks

This approach comes unfortunately with several caveats. The runtime of the program that the lib reloader operates in is a compiled Rust application with everything that involves. The memory layout of all types such as structs, enums, call frames, static string slices etc is fixed at compile time. If it diverges between the running application and the (re-)loaded library the effect is undefined. In that case the program will probably segfault but it might keep running with incorrect behavior. Because of that, all use of the reloader should be considered unsafe and I would highly advise against using it in situations where correctness and robustness of your application is required. This means, don’t use it for production environments or anything critical. The approach shown here is meant for development and fast feedback in situations where a crash is OK and nothing is at risk.

Rust nightly

Currently the macro for reading the library functions needs the proc_macro::Span feature in order to access the source_file. This is only available on Rust nightly so when you use hot-reloading you will need to switch to the nightly channel. If your app does not depend on nightly otherwise and you use a feature to enable hot-reload (discussed below) then you only need to use nightly when actually running with that feature. For example via cargo +nightly watch -i lib -x 'run --features reload'.

Global state

Because of that, what can actually be changed while maintaining the dynamic library re-loadable is limited. You can expose functions with signatures that shouldn’t change or static global variables. If exposing structs and enums, changing their struct fields is undefined behavior. Using generics across the dynamic library border isn’t possible either as generic signatures will always be mangled.

But state can get tricky in other ways as well. If global state is used by parts of a program that get reloaded, the state will no longer be accessible by the new code. This for example can be observed when trying to reload the macroquad / miniquad game frameworks. Those initialize and maintain OpenGL state globally. When trying to call reloaded code that uses the OpenGL context such as emitting render instructions, the program will panic. Even though it would be possible to externalize and transfer this state on reload and using the reloader with those frameworks won’t work, at least when directly accessing OpenGL related behavior. For strategies around this see How to use it then? below.

dlopen/dlclose behavior

Furthermore, as pointed out in this really great blog post from Amos the behavior of dlclose on Linux in combination with how Rust deals with thread local storage can actually prevent a library from properly unloading. Even though the approach presented here will actually allow you to load the changed behavior into your program, memory leaks cannot be prevented. So while you reload on Linux you will leak some (tiny) amount of memory on each library update.

Function signatures

The function names of reloadable functions need to be not mangled, so a #[no_mangle] attribute is mandatory. This in turn means that functions cannot be generic (with the exception of lifetimes). Assuming the rustc version of the code that compiles the binary and library are identical, the functions can be loaded even if not declared with extern "C". This in turn means that passing “normal” Rust types works fine. I do not know enough about how the Rust compiler chooses the memory layout of structs and enums to know if the same (non-generic) struct / enum definition can be laid out differently between compilations when the code remains unchanged. I assume not but if you run into problems making using extern "C" and #[repr(C)] might be worth trying.

Supported platforms

The current version of hot-lib-reloader has been tested successfully on Linux, Windows, and Intel macOS.

How to use it then?

The above mentioned limitations restrict the possibilities of what changes can be made to running code.

What typically works well is to expose one or a few functions whose signatures remain unchanged. Those can then be called from throughout your application. If you want to split up your code further, using multiple libraries and reloaders at the same time is no problem. The reloaders should regularly check if watched libraries have changed, for that purpose they provide an update method that returns true if the library was indeed reloaded. This gives you control over when reloads should happen and also allows you to run code to e.g. re-initialize state if the library changed.

For example, for developing a game a simple approach would be to expose a update(&mut State) function that modifies the game state and a render(&State) function that takes care of rendering that. Inside the game loop you call the update function of the lib reloader to process code changes and optionally re-initialize the game state.

For frameworks such as Bevy where the main loop is controlled elsewhere, you can normally register callbacks or event handlers that will be invoked. For dealing with Bevy systems in particular, see below.

Use features to toggle hot reload support

Since only few parts of the application need to differ between a hot reloadable and a static version, using Rust features is the recommended way to selectively use hot code reloading and to also produce production code without any dependencies on dynamic library loading.

The reload-feature example shows how you can modify the minimal code example presented above to achieve this.

If necessary, create indirections

In cases like miniquad where global state prevents reloading a library you can fall back to creating an indirection. For example if you want to have a render function that can be changed, you could use a function that decoratively constructs shapes / a scene graph that then gets rendered in the binary with calls that depend on OpenGL.

When you have full control over the main loop

Also, frameworks that follow a more “functional” style such as Bevy can be supported. For Bevy in particular there exists specific support in hot-lib-reloader, see the next section for details.

How to use it with Bevy

Bevy manages the programs main loop by default and allows the user to specify functions (which represent the “systems” in Bevy’s ECS implementation) which are then invoked while the game is running. It turns out this approach allows fits a hot-reloading mechanism really well. The system functions can be seamlessly provided by either a statically linked or dynamically loaded library. All input they need they can specify by function parameters. Bevy’s type-based “dependency injection” mechanism will then make sure to pass the right state to them. In addition, Bevy’s resource system is a good fit to manage the lib reloader instance. A separate system can deal with the update cycle.

You can find a ready-to-use example in the examples section of hot-lib-reloader. I will go through it step by step below.

The project structure can be identical to the example show above. We will rename lib to systems to make our intentions explicit:

$ tree
.
├── Cargo.toml
├── src
│   └── main.rs
└── systems
    ├── Cargo.toml
    └── src
        └── lib.rs

Bevy app binary

src/main.rs then defines the SystemsReloader which loads the functions provide by systems/src/lib.rs. Note the generate_bevy_systems: true which will be explained below:

#[cfg(feature = "reload")]
hot_lib_reloader::define_lib_reloader! {
    unsafe SystemsReloader {
        lib_name: "systems",
        source_files: ["../systems/src/lib.rs"],
        // This is important, it will generate reloadable system functions
        // in main.rs
        generate_bevy_systems: true,
    }
}

We can then define the main function that dose not differ much from othe Bevy apps. The hot-lib-reloader specific part is to also define a startup and an update system for the reloader.

use bevy::prelude::*;
use systems::*;

fn main() {
    let mut app = App::new();

    app.add_plugins(DefaultPlugins)
        .add_startup_system(setup)
        .add_system(player_movement_system)
        .add_system(bevy::window::close_on_esc);

    #[cfg(feature = "reload")]
    app.add_startup_system(setup_hot_reload)
        .add_system(update_lib);

    app.run();
}

Then implement the reloader specific systems (the other systems are loaded from the systems crate):

#[cfg(feature = "reload")]
struct LibLoaderUpdateTimer(Timer);

#[cfg(feature = "reload")]
pub fn setup_hot_reload(mut commands: Commands) {
    // Inject the SystemsReloader as a resource
    commands.insert_resource(SystemsReloader::new().expect("init lib loader"));
    // Define a timer that is used for triggering the reloader update
    commands.insert_resource(LibLoaderUpdateTimer(Timer::from_seconds(1.0, true)));
}

#[cfg(feature = "reload")]
fn update_lib(
    time: Res<Time>,
    mut lib: ResMut<SystemsReloader>,
    mut timer: ResMut<LibLoaderUpdateTimer>,
) {
    timer.0.tick(time.delta());
    if timer.0.finished() {
        lib.update().expect("update lib");
    }
}

Bevy systems

The two systems used in the example that are located in systems/src/lib.rs are run-of-the-mill bevy system functions that I’ll abbreviate here to save some room. The only notable piece here is the #[no_mangle] attribute of player_movement_system — and the absence of such an attribute with setup.

pub fn setup(mut commands: Commands) { /*...*/ }

#[no_mangle]
pub fn player_movement_system(
    keyboard_input: Res<Input<KeyCode>>,
    mut query: Query<&mut Transform, With<Player>>,
) { /*...*/ }

Only public functions with #[no_mangle] will be recognized as reloadable functions by the define_lib_reloader! macro above. By setting the generate_bevy_systems: true flag, the macro will not only implement a method SystemsReloader::player_movement_system but it will also generate a function like

pub fn player_movement_system(
    loader: Res<SystemsReloader>,
    keyboard_input: Res<Input<KeyCode>>,
    mut query: Query<&mut Transform, With<Player>>,
) {
    loader.player_movement_system(keyboard_input, query, time);
}

in src/main.rs. This functions implements the boilerplate that you would otherwise have to write by hand: Forward the system call to the library function as Bevy itself does not know that those should be called from a dynamic library. In combination with use systems::*;, this allows to seamlessly toggle the lib reloader using the reload feature:

If it is on, the unmangled system functions will be generated in main.rs and the app init will pick up those functions instead of the ones exported via systems::*. Also functions that don’t have the #[no_mangle] attribute will not be replaced this way, they will just be available in the statically linked variant. This allows setup to be used directly. It only runs once and does not need to be loaded throght the library.

When the feature is not present then all system functions are just loaded through the use statement and nothing is invoked dynamically.

In practice this works quite well and provides a boilerplate-free and quite flexible reload solution that allows you to selectively choose what code to reload and what not.

To build both the executable and reloadable systems library you can start two cargo processes. Building the library:

# watch ./systems/, build the library
$ cargo watch -w systems -x 'build -p systems'

Building the executable (for the reason why Windows needs special treatment see this note):

# watch everything but ./systems and run the app
$ cargo watch -i systems -x 'run --features reload'
# Note: on windows use
$ env CARGO_TARGET_DIR=target-bin cargo watch -i systems -x 'run --features reload'

Using cargo watch for the excutable is not strictly necessary but it allows you to quickly change it as well when you make modifications that aren’t compatible with hot-reload.

Hot-reload Template for cargo-generate

To quickly setup new projects that come with the stuff presented here out of the box, you can use a cargo-generate template available at: https://github.com/rksm/rust-hot-reload.

Run cargo generate rksm/rust-hot-reload to setup a new project.

Comments and feedback

I just started to use hot-lib-reloader and so far I am surprised how well it works. It is not bringing Rust on-par with Lisp or Smalltalk but with code that requires lots of twiddling and changes it is enjoyable to use! Let me know what your impressions are (there is e.g. a post on reddit) and feel free to report bugs on Github. Over and out.

1. Although with or without built-in support, folks are able to achieve it regardsless. ↩︎

2. No need to type that up or anything. This and more examples can be found in the hot-lib-reloader example section. ↩︎