Day 44: Building my VMs with Docker
source link: http://jvns.ca/blog/2021/01/22/day-44--got-some-vms-to-start-in-firecracker/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Another pretty short post today, still in Firecracker land.
decided to build my VMs with Docker instead of cloud-init
I’ve been trying to figure out how to build my VMs for a while. Previously my
plan was to just run cloud-init
when the VM started, because I already had
cloud-init.yaml
files to launch VMs that I’d written previously.
It turned out that for some reason I couldn’t get cloud-init
to start, and
also it felt like cloud-init
was going to be really slow to run – even when
it was failing, it was already taking more than 10 seconds. I really wanted the
VM to boot in less than 2 seconds.
So I decided to instead build my containers with Docker, convert the Docker filesystem to an ext4 image, and then start that image in Firecracker. Here’s what creating the image looks like in a bash script.
IMG_ID=$(docker build -q .)
CONTAINER_ID=$(docker run -td $IMG_ID /bin/bash)
MOUNTDIR=mnt
IMAGE=ubuntu.ext4
mount $IMAGE $MOUNTDIR
qemu-img create -f raw $IMAGE 800M
mkfs.ext4 $IMAGE
docker cp $CONTAINER_ID:/ $MOUNTDIR
It seems to work fine, and actually building my VMs with Docker feels a lot
simpler than doing it with cloud-init.yaml
, I think they might be easier to
develop this way.
setting up Docker-Compose’s bridges
I kind of want to run my VM management software with docker-compose
, but it
needs to make changes to the host network to set up the bridges / tap
interfaces.
I learned that Docker Compose by default creates a new bridge for every
docker-compose
file with a random name. I didn’t think this was going to
work for me, because I want to put my VMs on the same bridge as the gotty
container that needs to SSH into the VMs. So I needed to know the name of the
VM.
It turns out the Docker Compose is AMAZING and lets you explicitly set the name of the bridge for a network
So I set up a network in my docker-compose.yml
file that looks like this, and
put my gotty
container in the firenet
network.
version: "3.3"
networks:
firenet:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.101.0.0/16
driver_opts:
com.docker.network.bridge.name: firecracker0
I still haven’t tested this out because there are some other legos I need to put together, but I think it should work.
trying out fly.io
I got a bit frustrated with writing my own Firecracker VM service so later in the day, so I asked on Twitter if anyone knew of a cloud service where I could run a Firecracker VM. I was pretty sure the answer was no, but I’m happy I asked because I was wrong!
It turns out that https://fly.io supports a Docker container interface, but they actually just unpack the Docker container’s files, convert it into an ext4 filesystem, and boot a Firecracker VM with it. They don’t support you picking the init system or choosing a kernel and presumably the VMs are kind of locked down, but it’s not a container!
I’d initially assumed it was a container because Fargate and Kata Containers both run containers images in VMs by putting a container runtime inside the Firecracker VM and running the container that way.
I got some VMs to run on fly.io using their Go API after a few hours, so we’ll
see how that goes! I’ll do a few more experiments today. It was pretty
straightforward except that their API isn’t documented yet. But their command
line management tool is open source so I could just read the source for
flyctl
to figure out how it worked.
Recommend
-
24
Often I find myself needing a pristine Linux system for testing some program that is expected to work on a user’s machine with an environment that is possibly quite different to mine. I could spin up a virtual m...
-
6
Protecting VMs by tag with Rubrik REST APIs Published January 31, 2019 by Joshua Stenhouse 4
-
7
VMs which are not stretched in a stretched cluster, which policy to use? Duncan Epping · Dec 14, 2020 ·
-
16
vCLS VMs not powering on, insufficient resources error Duncan Epping · Nov 26, 2020 ·
-
21
K3S with MetalLB on Multipass VMs
-
7
I’ve been having a problem for a while where my virtual machines (that I use to set up the puzzles) don’t launch reliably – sometimes they work, and sometimes they don’t. I didn’t understand why this was before, and on Friday I think I...
-
9
Hello! On Tuesday I spent more time working on figuring out how to run VMs with Firecracker for my SSH game project. They still start super fast and I’m really excited about them. I got through 3 main things: learned 1 new...
-
3
On Tuesday I finally got around to do something I’d been meaning to do for a while: test how many Firecracker VMs I can actually run on my little DigitalOcean droplet with 1GB of RAM. This turned out to be less complicated to test than...
-
2
Yesterday I integrated my new device mapper code and spent a bunch of time trying to make my VMs boot faster and I learned a bunch of things. My plan for making them boot faster was: replace systemd with a lighter weight in...
-
6
What’s the Diff: VMs vs. Containers October 15, 2021 by Molly Clancy // 18 Comments
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK