Github GitHub - terraform-aws-modules/terraform-aws-eks: Terraform module to cre...
source link: https://github.com/terraform-aws-modules/terraform-aws-eks
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
terraform-aws-eks
A terraform module to create a managed Kubernetes cluster on AWS EKS. Available through the Terraform registry. Inspired by and adapted from this doc and its source code. Read the AWS docs on EKS to get connected to the k8s dashboard.
Assumptions
- You want to create an EKS cluster and an autoscaling group of workers for the cluster.
- You want these resources to exist within security groups that allow communication and coordination. These can be user provided or created within the module.
- You've created a Virtual Private Cloud (VPC) and subnets where you intend to put the EKS resources. The VPC satisfies EKS requirements.
Important note
The cluster_version
is the required variable. Kubernetes is evolving a lot, and each major version includes new features, fixes, or changes.
Always check Kubernetes Release Notes before updating the major version.
You also need to ensure your applications and add ons are updated, or workloads could fail after the upgrade is complete. For action, you may need to take before upgrading, see the steps in the EKS documentation.
An example of harming update was the removal of several commonly used, but deprecated APIs, in Kubernetes 1.16. More information on the API removals, see the Kubernetes blog post.
By default, this module manages the aws-auth
configmap for you (manage_aws_auth=true
). To avoid the following issue where the EKS creation is ACTIVE
but not ready, we implemented a retry logic with an local-exec
provisioner and wget
(by default) with failover to curl
.
If you want to manage your aws-auth
configmap, ensure you have wget
(or curl
) and /bin/sh
installed where you're running Terraform or set wait_for_cluster_cmd
and wait_for_cluster_interpreter
to match your needs.
For windows users, please read the following doc.
Usage example
A full example leveraging other community modules is contained in the examples/basic directory.
data "aws_eks_cluster" "cluster" { name = module.my-cluster.cluster_id } data "aws_eks_cluster_auth" "cluster" { name = module.my-cluster.cluster_id } provider "kubernetes" { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token load_config_file = false version = "~> 1.9" } module "my-cluster" { source = "terraform-aws-modules/eks/aws" cluster_name = "my-cluster" cluster_version = "1.17" subnets = ["subnet-abcde012", "subnet-bcde012a", "subnet-fghi345a"] vpc_id = "vpc-1234556abcdef" worker_groups = [ { instance_type = "m4.large" asg_max_size = 5 } ] }
Conditional creation
Sometimes you need to have a way to create EKS resources conditionally but Terraform does not allow to use count
inside module
block, so the solution is to specify argument create_eks
.
Using this feature and having manage_aws_auth=true
(the default) requires to set up the kubernetes provider in a way that allows the data sources to not exist.
data "aws_eks_cluster" "cluster" { count = var.create_eks ? 1 : 0 name = module.eks.cluster_id } data "aws_eks_cluster_auth" "cluster" { count = var.create_eks ? 1 : 0 name = module.eks.cluster_id } # In case of not creating the cluster, this will be an incompletely configured, unused provider, which poses no problem. provider "kubernetes" { host = element(concat(data.aws_eks_cluster.cluster[*].endpoint, [""]), 0) cluster_ca_certificate = base64decode(element(concat(data.aws_eks_cluster.cluster[*].certificate_authority.0.data, [""]), 0)) token = element(concat(data.aws_eks_cluster_auth.cluster[*].token, [""]), 0) load_config_file = false version = "1.10" } # This cluster will not be created module "eks" { source = "terraform-aws-modules/eks/aws" create_eks = false # ... omitted }
Other documentation
- Autoscaling: How to enable worker node autoscaling.
- Enable Docker Bridge Network: How to enable the docker bridge network when using the EKS-optimized AMI, which disables it by default.
- Spot instances: How to use spot instances with this module.
- IAM Permissions: Minimum IAM permissions needed to setup EKS Cluster.
- FAQ: Frequently Asked Questions
Doc generation
Code formatting and documentation for variables and outputs is generated using pre-commit-terraform hooks which uses terraform-docs.
Follow these instructions to install pre-commit locally.
And install terraform-docs
with go get github.com/segmentio/terraform-docs
or brew install terraform-docs
.
Contributing
Report issues/questions/feature requests on in the issues section.
Full contributing guidelines are covered here.
Change log
- The changelog captures all important release notes from v11.0.0
- For older release notes, refer to changelog.pre-v11.0.0.md
Authors
Created by Brandon O'Connor - [email protected]. Maintained by Max Williams and Thierno IB. BARRY. Many thanks to the contributors listed here!
License
MIT Licensed. See LICENSE for full details.
Requirements
Name Version terraform >= 0.13.1 aws >= 3.35.0 kubernetes >= 1.11.1 local >= 1.4 null >= 2.1 random >= 2.1 template >= 2.1Providers
Name Version aws >= 3.35.0 kubernetes >= 1.11.1 local >= 1.4 null >= 2.1 random >= 2.1 template >= 2.1Modules
Name Source Version fargate ./modules/fargate
node_groups ./modules/node_groups
Resources
Name Type aws_autoscaling_group.workers resource aws_autoscaling_group.workers_launch_template resource aws_cloudwatch_log_group.this resource aws_eks_cluster.this resource aws_iam_instance_profile.workers resource aws_iam_instance_profile.workers_launch_template resource aws_iam_openid_connect_provider.oidc_provider resource aws_iam_policy.cluster_elb_sl_role_creation resource aws_iam_role.cluster resource aws_iam_role.workers resource aws_iam_role_policy_attachment.cluster_AmazonEKSClusterPolicy resource aws_iam_role_policy_attachment.cluster_AmazonEKSServicePolicy resource aws_iam_role_policy_attachment.cluster_AmazonEKSVPCResourceControllerPolicy resource aws_iam_role_policy_attachment.cluster_elb_sl_role_creation resource aws_iam_role_policy_attachment.workers_AmazonEC2ContainerRegistryReadOnly resource aws_iam_role_policy_attachment.workers_AmazonEKSWorkerNodePolicy resource aws_iam_role_policy_attachment.workers_AmazonEKS_CNI_Policy resource aws_iam_role_policy_attachment.workers_additional_policies resource aws_launch_configuration.workers resource aws_launch_template.workers_launch_template resource aws_security_group.cluster resource aws_security_group.workers resource aws_security_group_rule.cluster_egress_internet resource aws_security_group_rule.cluster_https_worker_ingress resource aws_security_group_rule.cluster_primary_ingress_workers resource aws_security_group_rule.cluster_private_access resource aws_security_group_rule.workers_egress_internet resource aws_security_group_rule.workers_ingress_cluster resource aws_security_group_rule.workers_ingress_cluster_https resource aws_security_group_rule.workers_ingress_cluster_kubelet resource aws_security_group_rule.workers_ingress_cluster_primary resource aws_security_group_rule.workers_ingress_self resource kubernetes_config_map.aws_auth resource local_file.kubeconfig resource null_resource.wait_for_cluster resource random_pet.workers resource random_pet.workers_launch_template resource aws_ami.eks_worker data source aws_ami.eks_worker_windows data source aws_caller_identity.current data source aws_iam_instance_profile.custom_worker_group_iam_instance_profile data source aws_iam_instance_profile.custom_worker_group_launch_template_iam_instance_profile data source aws_iam_policy_document.cluster_assume_role_policy data source aws_iam_policy_document.cluster_elb_sl_role_creation data source aws_iam_policy_document.workers_assume_role_policy data source aws_iam_role.custom_cluster_iam_role data source aws_partition.current data source template_file.launch_template_userdata data source template_file.userdata data sourceInputs
Name Description Type Default Required attach_worker_cni_policy Whether to attach the Amazon managedAmazonEKS_CNI_Policy
IAM policy to the default worker IAM role. WARNING: If set false
the permissions must be assigned to the aws-node
DaemonSet pods via another method or nodes will not be able to join the cluster.
bool
true
no
aws_auth_additional_labels
Additional kubernetes labels applied on aws-auth ConfigMap
map(string)
{}
no
cluster_create_endpoint_private_access_sg_rule
Whether to create security group rules for the access to the Amazon EKS private API server endpoint.
bool
false
no
cluster_create_security_group
Whether to create a security group for the cluster or attach the cluster to cluster_security_group_id
.
bool
true
no
cluster_create_timeout
Timeout value when creating the EKS cluster.
string
"30m"
no
cluster_delete_timeout
Timeout value when deleting the EKS cluster.
string
"15m"
no
cluster_egress_cidrs
List of CIDR blocks that are permitted for cluster egress traffic.
list(string)
[no cluster_enabled_log_types A list of the desired control plane logging to enable. For more information, see Amazon EKS Control Plane Logging documentation (https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
"0.0.0.0/0"
]
list(string)
[]
no
cluster_encryption_config
Configuration block with encryption configuration for the cluster. See examples/secrets_encryption/main.tf for example format
list(object({
provider_key_arn = string
resources = list(string)
}))
[]
no
cluster_endpoint_private_access
Indicates whether or not the Amazon EKS private API server endpoint is enabled.
bool
false
no
cluster_endpoint_private_access_cidrs
List of CIDR blocks which can access the Amazon EKS private API server endpoint.
list(string)
null
no
cluster_endpoint_public_access
Indicates whether or not the Amazon EKS public API server endpoint is enabled.
bool
true
no
cluster_endpoint_public_access_cidrs
List of CIDR blocks which can access the Amazon EKS public API server endpoint.
list(string)
[no cluster_iam_role_name IAM role name for the cluster. If manage_cluster_iam_resources is set to false, set this to reuse an existing IAM role. If manage_cluster_iam_resources is set to true, set this to force the created role name.
"0.0.0.0/0"
]
string
""
no
cluster_log_kms_key_id
If a KMS Key ARN is set, this key will be used to encrypt the corresponding log group. Please be sure that the KMS Key has an appropriate key policy (https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/encrypt-log-data-kms.html)
string
""
no
cluster_log_retention_in_days
Number of days to retain log events. Default retention - 90 days.
number
90
no
cluster_name
Name of the EKS cluster. Also used as a prefix in names of related resources.
string
n/a
yes
cluster_security_group_id
If provided, the EKS cluster will be attached to this security group. If not given, a security group will be created with necessary ingress/egress to work with the workers
string
""
no
cluster_service_ipv4_cidr
service ipv4 cidr for the kubernetes cluster
string
null
no
cluster_version
Kubernetes version to use for the EKS cluster.
string
n/a
yes
config_output_path
Where to save the Kubectl config file (if write_kubeconfig = true
). Assumed to be a directory if the value ends with a forward slash /
.
string
"./"
no
create_eks
Controls if EKS resources should be created (it affects almost all resources)
bool
true
no
create_fargate_pod_execution_role
Controls if the EKS Fargate pod execution IAM role should be created.
bool
true
no
eks_oidc_root_ca_thumbprint
Thumbprint of Root CA for EKS OIDC, Valid until 2037
string
"9e99a48a9960b14926bb7f3b02e22da2b0ab7280"
no
enable_irsa
Whether to create OpenID Connect Provider for EKS to enable IRSA
bool
false
no
fargate_pod_execution_role_name
The IAM Role that provides permissions for the EKS Fargate Profile.
string
null
no
fargate_profiles
Fargate profiles to create. See fargate_profile
keys section in fargate submodule's README.md for more details
any
{}
no
iam_path
If provided, all IAM roles will be created on this path.
string
"/"
no
kubeconfig_aws_authenticator_additional_args
Any additional arguments to pass to the authenticator such as the role to assume. e.g. ["-r", "MyEksRole"].
list(string)
[]
no
kubeconfig_aws_authenticator_command
Command to use to fetch AWS EKS credentials.
string
"aws-iam-authenticator"
no
kubeconfig_aws_authenticator_command_args
Default arguments passed to the authenticator command. Defaults to [token -i $cluster_name].
list(string)
[]
no
kubeconfig_aws_authenticator_env_variables
Environment variables that should be used when executing the authenticator. e.g. { AWS_PROFILE = "eks"}.
map(string)
{}
no
kubeconfig_name
Override the default name used for items kubeconfig.
string
""
no
manage_aws_auth
Whether to apply the aws-auth configmap file.
bool
true
no
manage_cluster_iam_resources
Whether to let the module manage cluster IAM resources. If set to false, cluster_iam_role_name must be specified.
bool
true
no
manage_worker_iam_resources
Whether to let the module manage worker IAM resources. If set to false, iam_instance_profile_name must be specified for workers.
bool
true
no
map_accounts
Additional AWS account numbers to add to the aws-auth configmap. See examples/basic/variables.tf for example format.
list(string)
[]
no
map_roles
Additional IAM roles to add to the aws-auth configmap. See examples/basic/variables.tf for example format.
list(object({
rolearn = string
username = string
groups = list(string)
}))
[]
no
map_users
Additional IAM users to add to the aws-auth configmap. See examples/basic/variables.tf for example format.
list(object({
userarn = string
username = string
groups = list(string)
}))
[]
no
node_groups
Map of map of node groups to create. See node_groups
module's documentation for more details
any
{}
no
node_groups_defaults
Map of values to be applied to all node groups. See node_groups
module's documentation for more details
any
{}
no
permissions_boundary
If provided, all IAM roles will be created with this permissions boundary attached.
string
null
no
subnets
A list of subnets to place the EKS cluster and workers within.
list(string)
n/a
yes
tags
A map of tags to add to all resources. Tags added to launch configuration or templates override these values for ASG Tags only.
map(string)
{}
no
vpc_id
VPC where the cluster and workers will be deployed.
string
n/a
yes
wait_for_cluster_cmd
Custom local-exec command to execute for determining if the eks cluster is healthy. Cluster endpoint will be available as an environment variable called ENDPOINT
string
"for i in
seq 1 60; do if
command -v wget > /dev/null`; then wget --no-check-certificate -O - -q $ENDPOINT/healthz >/dev/null && exit 0
wait_for_cluster_interpreter
Custom local-exec command line interpreter for the command to determining if the eks cluster is healthy.
list(string)
[no worker_additional_security_group_ids A list of additional security group ids to attach to worker instances
"/bin/sh",
"-c"
]
list(string)
[]
no
worker_ami_name_filter
Name filter for AWS EKS worker AMI. If not provided, the latest official AMI for the specified 'cluster_version' is used.
string
""
no
worker_ami_name_filter_windows
Name filter for AWS EKS Windows worker AMI. If not provided, the latest official AMI for the specified 'cluster_version' is used.
string
""
no
worker_ami_owner_id
The ID of the owner for the AMI to use for the AWS EKS workers. Valid values are an AWS account ID, 'self' (the current account), or an AWS owner alias (e.g. 'amazon', 'aws-marketplace', 'microsoft').
string
"amazon"
no
worker_ami_owner_id_windows
The ID of the owner for the AMI to use for the AWS EKS Windows workers. Valid values are an AWS account ID, 'self' (the current account), or an AWS owner alias (e.g. 'amazon', 'aws-marketplace', 'microsoft').
string
"amazon"
no
worker_create_cluster_primary_security_group_rules
Whether to create security group rules to allow communication between pods on workers and pods using the primary cluster security group.
bool
false
no
worker_create_initial_lifecycle_hooks
Whether to create initial lifecycle hooks provided in worker groups.
bool
false
no
worker_create_security_group
Whether to create a security group for the workers or attach the workers to worker_security_group_id
.
bool
true
no
worker_groups
A list of maps defining worker group configurations to be defined using AWS Launch Configurations. See workers_group_defaults for valid keys.
any
[]
no
worker_groups_launch_template
A list of maps defining worker group configurations to be defined using AWS Launch Templates. See workers_group_defaults for valid keys.
any
[]
no
worker_security_group_id
If provided, all workers will be attached to this security group. If not given, a security group will be created with necessary ingress/egress to work with the EKS cluster.
string
""
no
worker_sg_ingress_from_port
Minimum port number from which pods will accept communication. Must be changed to a lower value if some pods in your cluster will expose a port lower than 1025 (e.g. 22, 80, or 443).
number
1025
no
workers_additional_policies
Additional policies to be added to workers
list(string)
[]
no
workers_egress_cidrs
List of CIDR blocks that are permitted for workers egress traffic.
list(string)
[no workers_group_defaults Override default values for target groups. See workers_group_defaults_defaults in local.tf for valid keys.
"0.0.0.0/0"
]
any
{}
no
workers_role_name
User defined workers role name.
string
""
no
write_kubeconfig
Whether to write a Kubectl config file containing the cluster configuration. Saved to config_output_path
.
bool
true
no
Outputs
Name Description cloudwatch_log_group_arn Arn of cloudwatch log group created cloudwatch_log_group_name Name of cloudwatch log group created cluster_arn The Amazon Resource Name (ARN) of the cluster. cluster_certificate_authority_data Nested attribute containing certificate-authority-data for your cluster. This is the base64 encoded certificate data required to communicate with your cluster. cluster_endpoint The endpoint for your EKS Kubernetes API. cluster_iam_role_arn IAM role ARN of the EKS cluster. cluster_iam_role_name IAM role name of the EKS cluster. cluster_id The name/id of the EKS cluster. Will block on cluster creation until the cluster is really ready cluster_oidc_issuer_url The URL on the EKS cluster OIDC Issuer cluster_primary_security_group_id The cluster primary security group ID created by the EKS cluster on 1.14 or later. Referred to as 'Cluster security group' in the EKS console. cluster_security_group_id Security group ID attached to the EKS cluster. On 1.14 or later, this is the 'Additional security groups' in the EKS console. cluster_version The Kubernetes server version for the EKS cluster. config_map_aws_auth A kubernetes configuration to authenticate to this EKS cluster. fargate_iam_role_arn IAM role ARN for EKS Fargate pods fargate_iam_role_name IAM role name for EKS Fargate pods fargate_profile_arns Amazon Resource Name (ARN) of the EKS Fargate Profiles. fargate_profile_ids EKS Cluster name and EKS Fargate Profile names separated by a colon (:). kubeconfig kubectl config file contents for this EKS cluster. kubeconfig_filename The filename of the generated kubectl config. node_groups Outputs from EKS node groups. Map of maps, keyed by var.node_groups keys oidc_provider_arn The ARN of the OIDC Provider ifenable_irsa = true
.
security_group_rule_cluster_https_worker_ingress
Security group rule responsible for allowing pods to communicate with the EKS cluster API.
worker_iam_instance_profile_arns
default IAM instance profile ARN for EKS worker groups
worker_iam_instance_profile_names
default IAM instance profile name for EKS worker groups
worker_iam_role_arn
default IAM role ARN for EKS worker groups
worker_iam_role_name
default IAM role name for EKS worker groups
worker_security_group_id
Security group ID attached to the EKS workers.
workers_asg_arns
IDs of the autoscaling groups containing workers.
workers_asg_names
Names of the autoscaling groups containing workers.
workers_default_ami_id
ID of the default worker group AMI
workers_launch_template_arns
ARNs of the worker launch templates.
workers_launch_template_ids
IDs of the worker launch templates.
workers_launch_template_latest_versions
Latest versions of the worker launch templates.
workers_user_data
User data of worker groups
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK