AWS ACK (AWS Controller for Kubernetes)
Modern kubernetes application often require additional infrastructure like Database, storage bucket etc. Often these resources are provisioned by Platform team and developments teams to coordinate infrastructure creation before deploying application. Tools like Crossplane and Kubernetes Resource Orchestrator aim to make it possible to manage it via Kubernetes object. KRO does this by using providers, and for AWS it is `ACK (AWS Controller for Kubernetes).
AWS Controllers for Kubernetes (ACK) lets you define and use AWS service resources directly from Kubernetes. ACK can be used directly on EKS/Kubernetes instances or along with other tools like KRO.
Here is how you can use ACK on an EKS cluster to provision AWS resources. Default documentation at present advise use of IAM RSA for controllers, I am using newer Pod Identity instead.
Step 1. Provision EKS-Auto mode and EKS with Pod identity enabled
To follow along you will need an EKS cluster with PodIdentity enables as I am using Pod Identity instead of IAM-RSA. Source Code in terraform is available in my Gitrepo https://github.com/vettom/aws-eks-terraform/EKS-Auto-ACK.
Step 2. Create IAM role with POD-ID for IAM-Chart
Each ACK service controller is packaged into a separate container image that is published in a public repository corresponding to an individual ACK service controller. Container images for ACK service controllers can be found in the ACK registry within the Amazon ECR Public Gallery that contains images as well Helm chart for the controller.
In this example, I am starting with IAM-CHART and it requires an IAM policy attachment with necessary permission to create IAM roles. Here is a sample policy with Pod identity configuration using terraform.
iam-chart-por-id.tf
data "aws_iam_policy_document" "pod_id_assume_role" {
statement {
effect = "Allow"
principals {
type = "Service"
identifiers = ["pods.eks.amazonaws.com"]
}
actions = [
"sts:AssumeRole",
"sts:TagSession"
]
}
}
resource "aws_iam_role" "ack-iam-controller" {
name = "ack-iam-controller"
assume_role_policy = data.aws_iam_policy_document.pod_id_assume_role.json
}
resource "aws_iam_policy" "ack-iam-controller" {
name = "ack-iam-controller-policy"
description = "Policy for ACK IAM Controller"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = [
"iam:CreateRole",
"iam:DeleteRole",
"iam:AttachRolePolicy",
"iam:DetachRolePolicy",
"iam:PutRolePolicy",
"iam:DeleteRolePolicy",
"iam:PassRole",
"iam:GetRole",
"iam:ListRolePolicies",
"iam:ListAttachedRolePolicies",
"iam:ListRoles",
"iam:TagRole",
"iam:UntagRole",
"iam:UpdateAssumeRolePolicy",
"iam:UpdateRole",
"iam:UpdateRoleDescription",
"iam:GetRolePolicy",
"iam:SimulatePrincipalPolicy",
"iam:ListRoleTags"
]
Effect = "Allow"
Resource = "*"
},
]
})
}
resource "aws_iam_role_policy_attachment" "ack-iam-controller" {
role = aws_iam_role.ack-iam-controller.name
policy_arn = aws_iam_policy.ack-iam-controller.arn
}
resource "aws_eks_pod_identity_association" "ack-iam-controller" {
cluster_name = module.eks.cluster_name
namespace = "ack-system"
service_account = "ack-iam-controller"
role_arn = aws_iam_role.ack-iam-controller.arn
}
Step 3. Provision IAM controller chart
IAM-Chart requires aws.region to be set.
export SERVICE=iam
export RELEASE_VERSION=$(curl -sL https://api.github.com/repos/aws-controllers-k8s/${SERVICE}-controller/releases/latest | jq -r '.tag_name | ltrimstr("v")')
export ACK_SYSTEM_NAMESPACE=ack-system
export AWS_REGION=eu-west-1
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install --create-namespace -n $ACK_SYSTEM_NAMESPACE ack-$SERVICE-controller \
oci://public.ecr.aws/aws-controllers-k8s/$SERVICE-chart --version=$RELEASE_VERSION --set=aws.region=$AWS_REGION
Step 4. Create sample IAM role
Create example-role and validate role is created in AWS.
apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
name: example-role
spec:
name: example-role
assumeRolePolicyDocument: |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
description: "Example role created by ACK"
maxSessionDuration: 3600
policies:
- arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
Step 5. Provisioning Additional controllers
Provisioning IAM-Controller first enables you to install additional controllers completely from Kubernetes. Here is example of S3 controller - Define IAM role with Pod-Identity association for S3 controller - Install S3 controller chart
Create IAM Pod ID role for S3 controller
iam-role-s3-podid.yaml
apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
name: ack-s3-controller
spec:
name: ack-s3-controller
assumeRolePolicyDocument: |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
description: "IAM role created by ACK for S3 Controller"
maxSessionDuration: 3600
policies:
- arn:aws:iam::aws:policy/AmazonS3FullAccess
Install S3 controller
export SERVICE=s3
export RELEASE_VERSION=$(curl -sL https://api.github.com/repos/aws-controllers-k8s/${SERVICE}-controller/releases/latest | jq -r '.tag_name | ltrimstr("v")')
export ACK_SYSTEM_NAMESPACE=ack-system
export AWS_REGION=eu-west-1
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install --create-namespace -n $ACK_SYSTEM_NAMESPACE ack-$SERVICE-controller \
oci://public.ecr.aws/aws-controllers-k8s/$SERVICE-chart --version=$RELEASE_VERSION --set=aws.region=$AWS_REGION