Tiny and Fast Docker image for Rust Application

Jun 7, 2021 , rev. Jun 14, 2021 - 8 minutes read

The steps below will use Rocket "hello world" application as a demo.

The basic configuration would be:

# Dockerfile.plain

ARG BASE_IMAGE=rust:1.52.1-slim-buster

FROM $BASE_IMAGE
WORKDIR app
COPY . .
RUN cargo build --release
CMD ["./target/release/hello"]

Let's build the image:

$ time docker build -f Dockerfile.plain -t hello:0.1.0 .

It produces 1.38GB image with 12 minutes build times. The second and the third build also needs a similar running time.

# second build
real    12m13.412s
user    0m0.135s
sys     0m0.095s

# third build
real    12m51.086s
user    0m0.118s
sys     0m0.062s

REPOSITORY    TAG                  IMAGE ID       CREATED          SIZE
hello         0.1.0                ac4e1a72ba05   2 minutes ago    1.38GB
rust          1.52.1-slim-buster   61cb3c65a6ba   3 weeks ago      621MB

I will test the build size output and duration three times. The first build will not be counted. I assume it will be a warm-up. Before the second and third steps are run, I changed the source code, from Hello, world! to Hello, world!1 and so on. To see if the current step has a caching mechanism. The test is held on a very quiet machine.

Let's use a multi-stage build to make the size smaller.

# Dockerfile.multistage

ARG BASE_IMAGE=rust:1.52.1-slim-buster

FROM $BASE_IMAGE as builder
WORKDIR app
COPY . .
RUN cargo build --release
CMD ["./target/release/hello"]

FROM $BASE_IMAGE
COPY --from=builder /app/target/release/hello /
CMD ["./hello"]

Now our image size is 628MB

# second test
real    11m42.625s
user    0m0.108s
sys     0m0.104s

# third test
real    11m38.789s
user    0m0.127s
sys     0m0.075s

REPOSITORY  TAG                  IMAGE ID       CREATED          SIZE
hello       0.1.0                e1fa9144e345   10 minutes ago   628MB

The second and the third build run at the same duration as the first.

I heard that cargo-chef able to use speed up Rust docker build, thanks to Docker layer caching. Let's use it.

# Dockerfile.chef

ARG BASE_IMAGE=rust:1.52.1-slim-buster

FROM $BASE_IMAGE as planner
WORKDIR app
RUN cargo install cargo-chef --version 0.1.20
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM $BASE_IMAGE as cacher
WORKDIR app
RUN cargo install cargo-chef --version 0.1.20
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM $BASE_IMAGE as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
COPY --from=cacher $CARGO_HOME $CARGO_HOME
RUN  cargo build --release

FROM $BASE_IMAGE
COPY --from=builder /app/target/release/hello /
CMD ["./hello"]

It produces a 628MB image size and needs 1 minute for the last build.

# first test omitted
# second test
real    19m32.836s
user    0m0.188s
sys     0m0.088s

# third test
real    1m52.982s
user    0m0.060s
sys     0m0.088s

REPOSITORY   TAG    IMAGE ID       CREATED              SIZE
hello        0.1.0  ca1a9e1e5948   About a minute ago   628MB

The last was far compared to its previous build. It will not happen without the help of cargo-chef. Using only multistage build, the last build still does a lot of work such as fetching the repo and compiling the crates.

The build will be faster if we could avoid the step for building cargo-chef. Most of my CI builds fetch the pre-built binary then copy it to the executable path such as ~/.cargo/bin. This trick help reduces one of my projects builds from ~30 minutes down to ~14 minutes.

A glimpse of the steps I described above:

# use pre-build trunk
wget -qO- https://github.com/thedodd/trunk/releases/download/v0.10.0/trunk-x86_64-unknown-linux-gnu.tar.gz | tar -xzf-
chmod +x trunk
cp trunk ~/.cargo/bin/
trunk --version

Now, we have a very fast build, but with a huge image. We could improve the situation with either one of the two ways. Use a distroless image, or use the scratch. But the latter requires you to change the toolchain target to musl.

Let's try using the scratch image first.

# Dockerfile.musl

ARG BASE_IMAGE=rust:1.52.1-slim-buster

FROM $BASE_IMAGE as planner
WORKDIR app
RUN cargo install cargo-chef --version 0.1.20
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM $BASE_IMAGE as cacher
WORKDIR app
RUN cargo install cargo-chef --version 0.1.20
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM $BASE_IMAGE as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
COPY --from=cacher $CARGO_HOME $CARGO_HOME
# We need static linking for musl
RUN rustup target add x86_64-unknown-linux-musl
# `cargo build` doesn't work in static linking, need `cargo install`
RUN cargo install --target x86_64-unknown-linux-musl --path .

FROM scratch
COPY --from=builder /usr/local/cargo/bin/hello .
CMD ["./hello"]

# first test ommitted
# second test
real    9m22.049s
user    0m0.113s
sys     0m0.103s

# third test
real    9m46.035s
user    0m0.120s
sys     0m0.085s

REPOSITORY   TAG    IMAGE ID       CREATED          SIZE
hello        0.1.0  332ce3b4f717   30 seconds ago   8.38MB

Now, let's try the distroless way.

# Dockerfile.distroless

ARG BASE_IMAGE=rust:1.52.1-slim-buster

FROM $BASE_IMAGE as planner
WORKDIR app
RUN cargo install cargo-chef --version 0.1.20
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM $BASE_IMAGE as cacher
WORKDIR app
RUN cargo install cargo-chef --version 0.1.20
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM $BASE_IMAGE as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
COPY --from=cacher $CARGO_HOME $CARGO_HOME
RUN cargo build --release

FROM gcr.io/distroless/cc-debian10
COPY --from=builder /app/target/release/hello /
CMD ["./hello"]

# first test ommitted
# second test
real    1m57.033s
user    0m0.045s
sys     0m0.038s

# third test
real    2m12.585s
user    0m0.093s
sys     0m0.096s

REPOSITORY   TAG    IMAGE ID       CREATED          SIZE
hello        0.1.0  118da37bdfe7   25 seconds ago   29MB

Using musl and scratch produces the smallest image 8.38MB. Compared to distroless 29MB. But the rebuild is faster in the distroless side 2m compared to musl 9m.

I made the repo containing each of the Dockerfile under the test for you to play with.

Conclusion #

The first try produces a 1.38GB image size with 12m build times. Now we can produce a very small image with a very fast build.

The choice is in your hand. If you are ok with musl target, go with scratch image. Otherwise, choose distroless.

Notes #

Thanks to Mikail Bagishov for telling me about the distroless image.

On 2021-06-10 Pieter S. van N told me about docker-slim. I tried it right away. I need to set --http-probe=false for both scratch and distroless image. Otherwise, it will fail. The scratch can not be optimized anymore. However, the distroless image can be minified by 2.46X.

I tested the minified image and it works well for this simple app. I don't have any clue for a large and complex app if the minification has any side-effect. The distroless image now has a comparable size with the scratch image without sacrificing the target changes to musl. Thanks to docker-slim.

REPOSITORY               TAG      IMAGE ID       CREATED         SIZE
hello/musl.slim          latest   54832fa90141   4 hours ago     8.53MB
hello/musl               latest   a7f6ff9598b6   10 hours ago    8.53MB
hello/distroless.slim    latest   312e081e8f63   27 seconds ago  11.9MB
hello/distroless         latest   0e434919836a   10 hours ago    29.2MB

My previous attempt to use docker-slim with a tagged image is failed. I thought doker-slim can't play well with tags. Turns out it's the http-probe issue. I think the error message is misleading. It says image not found. So I thought it was a tag issue.

cmd=build info=target.image.error status='image.not.found' image='hello/musl:1.0.1' message='make sure the target image already exists locally'

If you liked this article, please support my work. It will definitely be rewarding and motivating. Thanks for the support!