Practical Implementation: Setup Containerized Build Agents

Hello, readers! This article talks about the Practical Implementation to Setup a Containerized Build Agent in any Kubernetes Environment in detail.

So, let us begin!! 🙂

What is a Build Agent?

In order to build and produce any software or a product, we usually design steps and also list down the artifacts and libraries necessary to build the software from the very initial stage.

In the early days, when Virtual Machines and on-premises servers were in the lead, they were primarily responsible for building the software, maintaining databases, etc. This sounds like a huge pile of tasks instructed to be executed by a single workforce.

This is when the need for Build Agent emerged.

The build is the process that usually accepts the source code and then builds steps over it towards the end product. Build actually reads and downloads the source code to the Build agent machine and makes use of the libraries/software downloaded on that particular machine to achieve the end goal. As an end result, it generates a build status and a build artifact.

These build agents can be some on-premise machine or a Virtual machine or even containers. An Agent pool is an abstraction level provided by Azure DevOps. It isolates our build agents for a particular set of tasks. So, the moment we execute a build pipeline targeting a particular agent pool, it gets executed on an agent from the defined Agent pool.

Now, in the world of containers, we can even containerize a build agent. That is, we can host our build agents as containers instead of setting them up as a whole Virtual Machine.

The build agents will run as containers in the Kubernetes environment and serve the same purpose from within the Agent Pool.

Let us have a look at some of the benefits of Containerized build agents in the upcoming section.

Benefit of Containerized Build agent

It enables Cost Optimization. If the builds are not continuous, Kubernetes will scale down the resources of the containerized build agents. On the other hand, if we make use of Virtual Machines as Build agents, they keep consuming resources even when not in use.
Containerized Build Agents enable us to optimize resources as well. We do not need to set up static Virtual machines as build agents. Instead, ephemeral containers can be set up to reduce the carbon footprint as well.
Spinning up containers is a task that takes just a couple of minutes. Thus, we can easily spin up containers without having to have the entire background setup up and functional.

Steps to Containerize a Build Agent

In order to set up build agents as containers, we need to follow the below steps-

Create an Agent Pool in Azure DevOps.
Create or set up an Image Container Registry in Azure or Google or any private registry of your choice.
Build the script for Build agent and generate an Image to be pushed to the Container Registry.
Generate a Secret to connect to Azure DevOps agent pool.
Create a Deployment YAML file to containerize the build agent.
Confirm the Build agents being deployed as containers in the Kubernetes cluster.

1. Creation of an Agent Pool in Azure DevOps

Login to Azure DevOps and click on Settings. Further, Click on Agent Pools under the Pipeline menu. Click on Add Pool as shown below-

Azure DevOps – Agent Pool

Agent Pool Creation

2. Create a Container Registry

Create a Container Registry of your choice. For the purpose of demonstration, we have chosen Azure Container Registry.

Container Registry

3. Build the Dockerfile to generate an Image

In order to create a containerized build agent, we make use of the below shell scripted offered to us by Azure-

build.sh

#!/bin/bash

set -e

if [ -z "$AZP_URL" ]; then

echo 1>&2 "error: missing AZP_URL environment variable"

exit 1

fi

if [ -z "$AZP_TOKEN_FILE" ]; then

if [ -z "$AZP_TOKEN" ]; then

echo 1>&2 "error: missing AZP_TOKEN environment variable"

exit 1

fi

AZP_TOKEN_FILE=/azp/.token

echo -n $AZP_TOKEN > "$AZP_TOKEN_FILE"

fi

unset AZP_TOKEN

if [ -n "$AZP_WORK" ]; then

mkdir -p "$AZP_WORK"

fi

rm -rf /azp/agent

mkdir /azp/agent

cd /azp/agent

export AGENT_ALLOW_RUNASROOT="1"

cleanup() {

if [ -e config.sh ]; then

print_header "Cleanup. Removing Azure Pipelines agent..."

./config.sh remove --unattended \

--auth PAT \

--token $(cat "$AZP_TOKEN_FILE")

fi

}

print_header() {

lightcyan='\033[1;36m'

nocolor='\033[0m'

echo -e "${lightcyan}$1${nocolor}"

}

# Let the agent ignore the token env variables

export VSO_AGENT_IGNORE=AZP_TOKEN,AZP_TOKEN_FILE

print_header "1. Determining matching Azure Pipelines agent..."

AZP_AGENT_RESPONSE=$(curl -LsS \

-u user:$(cat "$AZP_TOKEN_FILE") \

-H 'Accept:application/json;api-version=3.0-preview' \

"$AZP_URL/_apis/distributedtask/packages/agent?platform=linux-x64")

if echo "$AZP_AGENT_RESPONSE" | jq . >/dev/null 2>&1; then

AZP_AGENTPACKAGE_URL=$(echo "$AZP_AGENT_RESPONSE" \

fi

if [ -z "$AZP_AGENTPACKAGE_URL" -o "$AZP_AGENTPACKAGE_URL" == "null" ]; then

echo 1>&2

"error: could not determine a matching Azure Pipelines agent - check that account '$AZP_URL' is correct and the token is valid for that account"

exit 1

fi

print_header "2. Downloading and installing Azure Pipelines agent..."

curl -LsS $AZP_AGENTPACKAGE_URL | tar -xz & wait $!

source ./env.sh

print_header "3. Configuring Azure Pipelines agent..."

./config.sh --unattended \

--agent "${AZP_AGENT_NAME:-$(hostname)}" \

--url "$AZP_URL" \

--auth PAT \

--token $(cat "$AZP_TOKEN_FILE") \

--pool "${AZP_POOL:-Default}" \

--work "${AZP_WORK:-_work}" \

--replace \

--acceptTeeEula & wait $!

print_header "4. Running Azure Pipelines agent..."

trap 'cleanup; exit 130' INT

trap 'cleanup; exit 143' TERM

# To be aware of TERM and INT signals call run.sh

# Running it with the --once flag at the end will shut down the agent after the build is executed

./run.sh & wait $!

We then create a Dockerfile out of this script to generate an Image as shown below-

Dockerfile

FROM ubuntu:18.04

# To make it easier for build and release pipelines to run apt-get,

# configure apt to not require confirmation (assume the -y argument by default)

ENV DEBIAN_FRONTEND=noninteractive

RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes

RUN apt-get update \

&& apt-get install -y --no-install-recommends \

ca-certificates \

curl \

jq \

git \

iputils-ping \

libcurl4 \

libicu60 \

libunwind8 \

netcat \

libssl1.0

WORKDIR /azp

COPY ./build.sh .

RUN chmod +x build.sh

CMD ["./build.sh"]

With the above Dockerfile, we build a Docker Image and push it to a private container registry of our choice.

Post the build, we can review our image in the registry as follows-

Azure Container Registry

4. Creating a Secret to connect to Azure DevOps

In order to connect to Azure DevOps from within the Kubernetes cluster and deploy a build agent, we need to generate a secret and include the Azure DevOps Personal Access Token into it-

secret.YAML

apiVersion: v1

data:

AZP_TOKEN: Xo3cjRxNGJocWN0YWZhaHo3Ym11NnpjdTN5Z2EyY2U1cm5rN2M2NXd==

kind: Secret

metadata:

name: azure-devops-agent-secret

namespace: azure-ba

type: Opaque

5. Generate a Kubernetes Deployment file for Build Agent Deployment

As a final step, we need to generate a Kubernetes Deployment file to deploy the build agent image as a container within the Azure DevOps Agent Pool.

Deployment.YAML

apiVersion: apps/v1

kind: Deployment

metadata:

name: devops-agent

namespace: azure-ba

spec:

replicas: 2

selector:

matchLabels:

app: devops-agent

version: linux-1

template:

metadata:

labels:

app: devops-agent

version: linux-1

spec:

containers:

- image: demo.azurecr.io/demo14:latest

imagePullPolicy: Always

name: devops-agent-container

resources:

limits:

cpu: 1

memory: 1Gi

requests:

cpu: 1

memory: 1Gi

env:

- name: AZP_URL

value: "https://azure.visualstudio.com"

- name: AZP_TOKEN

valueFrom:

secretKeyRef:

name: azure-devops-agent-secret

key: AZP_TOKEN

- name: AZP_POOL

value: Demo-pool

imagePullSecrets:

- name: acr-secret

Inspect the Build agents as containers in Kubernetes environment

Once we deploy the above YAML file, our build agents should be seen running as containers in the respective Kubernetes environment-

NAME READY STATUS RESTARTS

devops-agent-8454b5dbd7-gsq4v 1/1 Running 1

devops-agent-8454b5dbd7-rfj7r 1/1 Running 0

Conclusion

By this, we have approached the end of this topic. Feel free to comment below, in case you come across any questions.

For more such posts related to Kubernetes and Azure DevOps, stay tuned with us.

Till then, Happy Learning!! 🙂

What is a Build Agent?

Benefit of Containerized Build agent

Steps to Containerize a Build Agent

1. Creation of an Agent Pool in Azure DevOps

2. Create a Container Registry

3. Build the Dockerfile to generate an Image

4. Creating a Secret to connect to Azure DevOps

5. Generate a Kubernetes Deployment file for Build Agent Deployment

Inspect the Build agents as containers in Kubernetes environment

Conclusion

Recommend

Understanding Gateway and Virtual Service in Istio

B站发布年度报告

Cloud SQL Proxy in Google Cloud Compute - A Complete Guide - JournalDev

Top stories of 2021: Q3

Xiaomi Black Shark 4S Pro keeps its AnTuTu title in December

相互宝将于1月底关停

商汤科技港交所上市

Pandas For Data Analysis - A Quick Guide - JournalDev

索尼将成立电动汽车公司

Oppo A96 5G leaked renders reveal dual camera, flat sides

About Joyk