GitLab CI runners in Kubernetes using spot/preemptible instances

Running your GitLab CI jobs using their shared runners is really simple and straightforward, but you may want to use your own runners for several reasons:

• Build minutes limits (currently 400 per month)
• Build performance: ability to use high-end CPU performant instances (I observed near x2 improvement on CPU-intensive builds duration)
• And also network performance (ie. when using your cloud provider's registry), configuration fine tuning, etc.

However, using your own runners may become quickly expensive when dedicating instances for this purpose, which will probably only be used partially (only when building, probably not at night, etc.).

If you are running Kubernetes in the cloud, on AWS, GCP or similar, you can leverage the usage of spot or preemptible instances to run your builds at low cost (up to the tenth of regular price). When coupled with cluster autoscaling, Nodes (instances) will be added to your cluster only when required, and stopped automatically.

Setup

In your existing Kubernetes cluster, add a Node Pool with the following settings:

• Labels: workload-type: gitlab-runner
• Taints: dedicated=gitlab-runner:NoSchedule
• Type: Spot (AWS) or Preemptible (GCP)

We'll be using Helm to deploy Gitlab Runner using official chart. To configure it, create a values.yaml file: 

gitlabUrl: https://gitlab.com/

runnerRegistrationToken: "<your-gitlab-runner-token>"

concurrent: 10

checkInterval: 5

rbac:
  create: true
  clusterWideAccess: false
  # serviceAccountName: default

runners:
  # Wait 10 minutes for runners availability, ie. when a node needs to be added
  pollTimeout: 600

  image: ubuntu:20.04

  # Required when using Docker-in-Docker
  privileged: true

  namespace: gitlab

  tags: "docker,kubernetes"

  builds:
    # cpuLimit: 200m
    # Be careful when setting memory limits, as runners will be killed when exceed it
    # memoryLimit: 256Mi
    cpuRequests: "1"
    memoryRequests: 1Gi

  services:
    cpuRequests: 100m
    memoryRequests: 128Mi

  helpers:
    cpuRequests: 100m
    memoryRequests: 128Mi

  # Force runners to run on specific nodes with this label
  nodeSelector:
    worlad-type: gitlab-runner

  # Allow runners to run on specific nodes with this taint
  nodeTolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "gitlab-runner"
      effect: "NoSchedule"

Then deploy the chart to your cluster:

# Using Helm 3

helm repo add gitlab https://charts.gitlab.io

helm upgrade --install --namespace gitlab gitlab-runner -f values.yaml gitlab/gitlab-runner

You're all set, you should now see your runner in your CI/CD GitLab settings, Runners section.