Use AAD Authentication for Pods running in AKS
source link: https://www.programmingwithwolfgang.com/use-aad-authentication-for-pods-running-in-aks/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Use AAD Authentication for Pods running in AKS
Since the dawn of time, authentication has been a problem for developers and security engineers. Weak passwords make it easy for attackers to get access to areas where they don’t belong. Even if you have a very long and complex password, you might be at risk. Passwords get leaked almost every day nowadays. The current recommendation for authentication is passwordless. This means that no password exists anymore and you authenticate with an authenticator app or a security key.
This approach is safe but not possible for applications. This is where Azure Active Directory authentication comes into play. Resources in Azure can authenticate using their identity and then get an access token from AAD. Using these identities is quite easy with standard resources like App Service or Key Vault but not as easy when using Kubernetes.
Today, I would like to show you how you can configure your Azure Kubernetes Service cluster to use AAD Pod Identity to authenticate against the AAD.
This post is part of “Microservice Series - From Zero to Hero”.
What are Managed Identities
Every application faces the same challenges when it comes to securing its credentials. Managed identities eliminate the need for credentials and help to make applications more secure at the same time. Since there are no passwords with managed identities, no password can be compromised which could give unauthorized people access to resources.
Managed identities can be used to access a database or an Azure Key Vault where the application can load all required credentials for resources that do not support managed identities yet. Additionally, managed identities can be used without any additional costs. There are two types of managed identities, system-assigned and user-assigned.
System-Assigned Managed Identity
Some resources in Azure like App Service or Azure Container Registry allow you to enable a managed identity directly on that service. This identity is created in the Azure AD and shares the same lifecycle as the service instance. When you delete the service, the system-assigned managed identity will be also deleted.
System assigned managed identity in ACR
User-Assigned Managed Identity
A user-assigned managed identity is a standalone resource in Azure. It can be assigned the resource and can be used in the same way as the system-assigned managed identity to authenticate that resource. The user-assigned managed identity has an independent lifecycle. This means that you can delete the Azure resource where the identity is assigned to and the identity does not get deleted.
Enable Managed Identities for Applications running in Azure Kubernetes Service (AKS)
Enabling managed identities for applications in AKS, so-called pod identities, is unfortunately not straightforward. There are different documentation but neither of them works reliably. You can find a demo integration on Github but the documentation is missing some steps and the scripts did not work for me (mostly due to wrong file paths).
Follow this guide to install pod identity on your AKS cluster and then test it with a demo application. Afterwards, I will go into the details of what is happening.
Set up your AKS Cluster for Pod Identities
For this tutorial, I will assume that you have no AKS cluster created yet. This demo uses the Azure CLI and bash. Before you begin, create a couple of variables, which you can use throughout this demo. Note to replace the value of SubscriptionId with the Id of your Azure Subscription.
export ResourceGroupName=MicroserviceDemo export AksClusterName=microservice-aks export NodeResourceGroupName=MC_MicroserviceDemo_microservice-aks_westeurope export location=westeurope export SubscriptionId=<YourAzureSubscriptionId> export ManagedIdentityName=managedidentity-aks
Set some variables
Next, create an AKS cluster with a managed identity and the azure network plugin.
az aks create -n $AksClusterName -g $ResourceGroupName --network-plugin azure --generate-ssh-keys --enable-managed-identity
After the AKS cluster is created, retrieve the client Id of the managed identity of the cluster. Then use this client Id to assign the Managed Identity Operator and Virtual Machine Contributor role to the managed identity.
export AksManagedIdentity="$(az aks show -g $ResourceGroupName -n $AksClusterName --query identityProfile.kubeletidentity.clientId -o tsv)" az role assignment create --role "Managed Identity Operator" --assignee $AksManagedIdentity --scope /subscriptions/$SubscriptionId/resourcegroups/$NodeResourceGroupName az role assignment create --role "Virtual Machine Contributor" --assignee $AksManagedIdentity --scope /subscriptions/$SubscriptionId/resourcegroups/$NodeResourceGroupName
This finishes the setup of the AKS cluster. Connect to your newly created cluster and let’s continue to configure the pod identity.
az aks get-credentials --resource-group $ResourceGroupName --name $AksClusterName
Configure Pod Identity in Azure Kubernetes Service
Install the AAD Pod Identity Helm chart using Helm. If you do not know Helm, see Helm - Getting Started for more information. Add and install the Helm chart with the following command:
helm repo add aad-pod-identity https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts helm install aad-pod-identity aad-pod-identity/aad-pod-identity
The AAD Pod Identity Helm Chart was installed successfully
After the installation is finished, you will see a message to verify that all associated pods have been started. Use the following command to check that:
kubectl --namespace=default get pods -l "app.kubernetes.io/component=mic" kubectl --namespace=default get pods -l "app.kubernetes.io/component=nmi"
This should show that all pods are running.
All AAD Pod Identity Pods are running
Next, create a new identity and export its client Id and Id:
az identity create -g $NodeResourceGroupName -n $ManagedIdentityName export ManagedIdentityClientId="$(az identity show -g $NodeResourceGroupName -n $ManagedIdentityName --query clientId -otsv)" export ManagedIdentityId="$(az identity show -g $NodeResourceGroupName -n $ManagedIdentityName --query id -otsv)"
Create a new Identity
With this identity, create a new Kubernetes object of the type AzureIdentity and add the identity Id and client Id to it.
cat <<EOF | kubectl apply -f - apiVersion: "aadpodidentity.k8s.io/v1" kind: AzureIdentity metadata: name: $ManagedIdentityName spec: type: 0 resourceID: $ManagedIdentityId clientID: $ManagedIdentityClientId EOF
Optionally, you could save the AzureIdentity in a file and then apply this file. If you are using this approach, you have to replace the variables with their actual values though.
The last step of the configuration of pod identities is to create an AzureIdentityBinding using the following command:
cat <<EOF | kubectl apply -f - apiVersion: "aadpodidentity.k8s.io/v1" kind: AzureIdentityBinding metadata: name: $ManagedIdentityName spec: azureIdentity: $ManagedIdentityName selector: $ManagedIdentityName EOF
Test the usage of the Pod Identity
Microsoft has a test application that will retrieve a token and give you a success message when the operation is completed. If something went wrong, you will see an error message. Deploy the test application with the following command:
cat << EOF | kubectl apply -f - apiVersion: v1 kind: Pod metadata: name: demo labels: aadpodidbinding: $ManagedIdentityName spec: containers: - name: demo image: mcr.microsoft.com/oss/azure/aad-pod-identity/demo:v1.8.4 args: - --subscription-id=$SubscriptionId - --resource-group=$NodeResourceGroupName - --identity-client-id=$ManagedIdentityClientId nodeSelector: kubernetes.io/os: linux EOF
Wait for around a minute and then check the logs of the pod. You can do this either using kubectl or a dashboard. Kubectl gives you a wall of text that might be hard to read.
kubectl logs demo
I prefer Octant as my dashboard. You can find more information about the usage and installation in Azure Kubernetes Service - Getting Started.
The token was retrieved successfully
What happens inside Kubernetes when adding Pod Identity
There was a lot going on during the demo which might be hard to understand at first. Let’s go through it step by step.
Azure Kubernetes Service Cluster installation
The AKS cluster is created with the flags –network-plugin azure and –enable-managed-identity. AKS supports two network modes, kubenet and azure, whereas kubenet is the default mode.
AAD Pod Identity is disabled by default on clusters using the Kubnet network mode. This is due to security concerns because Kubenet is susceptible to APR spoofing. This makes it possible for pods to impersonate other pods and gain access to resources which they are not allowed to access. The Azure network plugin prevents ARP Spoofing.
The second flag –enable-managed-identity creates a service principal. This service principal is used to authenticate against the AAD and retrieve tokens for the pods. Adding the Managed Identity Operator and Virtual Machine Contributor roles to the service principal is necessary so it can assign and un-assign identities from the worker nodes that run on the VM scale set.
Managed Identity Controller (MIC) and Node Managed Identity (NMI)
The Managed Identity Controller (MIC) monitors pods through the Kubernetes API Server. When the MIC detects any changes, it adds or deletes AzureAssignedIdentity if needed. Keep in mind that the identity is applied when a pod is scheduled. This means if you change your configuration, you have to re-schedule your pods for the changes to be applied.
The Node Managed Identity (NMI) intercepts traffic on all nodes and makes an Azure Active Directory Authentication Library (ADAL) request to get a token on behalf of the pods. The request to get a token is sent to the Azure Instance Metadata Service (IMDS) at 169.254.169.254. The answer is redirected to the NMI pod due to changes to the iptable rules of the nodes.
AzureIdentity and AzureIdendityBinding
AzureIdentity can be a user-assigned identity, service principal, or service principal with a certificate and is used to authenticate the requests from the NMI to get a token from the AAD.
The AzureIdendityBinding connects the AzureIdentity to a pod. The AzureIdentity is assigned to a pod if the selectors are matching.
The following screenshot shows the workflow of the token acquisition.
Token acquisation workflow (Github)
Uninstalling AAD Pod Identity
You can use the following code to uninstall pod identity.
kubectl delete pod demo kubectl delete azureidentity $ManagedIdentityName kubectl delete azureidentitybinding $ManagedIdentityName-binding az identity delete -g $NodeResourceGroupName -n $ManagedIdentityName
The iptables that got modified by the NMI pods should be cleaned up automatically when the pod identity pods are uninstalled. If the pods get terminated unexpectedly, the entries in the iptables are not removed. You can do that with the following commands:
iptables -t nat -D PREROUTING -j aad-metadata iptables -t nat -F aad-metadata iptables -t nat -X aad-metadata
Conclusion
Using AAD authentication in Azure Kubernetes Service is unfortunately not straightforward, especially since the documentation is missing some crucial information. Nevertheless, going passwordless is definitely worth the hassle and will make it easier to manage your applications in the future.
In my next post, I will show you how to use this AAD authentication to access your database without a password.
This post is part of “Microservice Series - From Zero to Hero”.
Recommend
-
12
AKS with Terraform on Rancher Shared as a Service
-
13
Accelerate containerized application development Easily define, deploy, debug, and upgrade even the most complex Kubernetes applications, and automatically containerize your applications. U...
-
9
Series for everyoneAKS Zero To Hero – Series for everyonePosted in AKS, Azure
-
7
Cluster auto-scaling for Azure Kubernetes Service (AKS) is available for quite some time now. I have been using it in several projects so far.This post explains all...
-
4
Relationship between OAuth2.0, AAD and MSALDefinitionsOAuth 2.0OAuth 2.0 is the industry-standard protocol for authorization. OAuth 2.0 focuses on client developer simplicity while providing specifi...
-
11
AKS vs EKS vs GKE: Managed Kubernetes services compared Alexander Potasnick Jun 9, 2021 15 Minute Read
-
5
Implement AAD Authentication to access Azure SQL Databases Oct 312021-11-01T00:00:00+01:00 by Wolfgang Ofner 13 minMicrosoft promotes going passwordless for a while now. Azure offers authenti...
-
4
Use AAD Authentication for Applications running in AKS to access Azure SQL Databases 2 days ago2021-12-06T00:00:00+01:00 by Wolfgang Ofner Removing passwords and using identities to access resources is t...
-
3
Reading Time: 5 minutes Prerequisites:- Kubernetes Cluster should be up and running. OVERVIEW Inside our Host Machine, we are setting up the
-
6
Kubernetes 1.25: alpha support for running Pods with user namespacesMonday, October 03, 2022Authors: Rodrigo Campos (Microsoft), Giuseppe Scrivano (Red Hat)Kubernetes v1.25 introduces the su...
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK