AWS Controller for Kubernetes(ACK) Getting started

EKS Auto mode ACK (Aws controller for Kubernetes) Terraform logo

Modern kubernetes application often require additional infrastructure like Database, storage bucket etc. Often these resources are provisioned by Platform team and developments teams to coordinate infrastructure creation before deploying application. Tools like Crossplane and Kubernetes Resource Orchestrator aim to make it possible to manage it via Kubernetes object. KRO does this by using providers, and for AWS it is `AWS Controller for Kubernetes aka ACK.

AWS Controllers for Kubernetes (ACK) lets you define and use AWS service resources directly from Kubernetes. ACK can be used directly on EKS/Kubernetes instances or along with other tools like KRO.

In this example, I will demonstrate how to install ACK controllers with IAM PodIdentity instead of IAM RSA

ACK (Aws controller for Kubernetes)

Step 1. Provision EKS-Auto mode and EKS with Pod identity enabled

To follow along you will need an EKS cluster with PodIdentity enables as I am using Pod Identity instead of IAM-RSA. Source Code in terraform is available in my Gitrepo https://github.com/vettom/aws-eks-terraform/eks-auto-ACK.

Step 2. Create IAM role with POD-ID for IAM-Chart

Each ACK service controller is packaged into a separate container image that is published in a public repository corresponding to an individual ACK service controller. Container images for ACK service controllers can be found in the ACK registry within the Amazon ECR Public Gallery that contains images as well Helm chart for the controller.

In this example, I am starting with IAM-CHART and it requires an IAM policy attachment with necessary permission to create IAM roles. Here is a sample policy with Pod identity configuration using terraform.
ack-role-iam-podid.tf

data "aws_iam_policy_document" "pod_id_assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["pods.eks.amazonaws.com"]
    }

    actions = [
      "sts:AssumeRole",
      "sts:TagSession"
    ]
  }
}

resource "aws_iam_role" "ack-iam-controller" {
  name               = "ack-iam-controller"
  assume_role_policy = data.aws_iam_policy_document.pod_id_assume_role.json
}

resource "aws_iam_policy" "ack-iam-controller" {
  name        = "ack-iam-controller-policy"
  description = "Policy for ACK IAM Controller"

  policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "AllowRoleCreation",
        "Effect": "Allow",
        "Action": [
          "iam:CreateRole",
          "iam:TagRole",
          "iam:PassRole",
          "iam:UpdateAssumeRolePolicy",
          "iam:GetRole",
          "iam:ListRolePolicies",
          "iam:PutRolePolicy",
          "iam:DeleteRole",
          "iam:DeleteRolePolicy",
          "iam:ListRoles",
          "iam:ListRoleTags",
          "iam:AttachRolePolicy",
          "iam:DetachRolePolicy",
          "iam:ListAttachedRolePolicies",
          "iam:ListPolicies",
          "iam:GetPolicy",
          "iam:GetPolicyVersion"          
        ],
        "Resource": "*"
      },
      {
        "Sid": "DenyAdministratorAccessAttachment",
        "Effect": "Deny",
        "Action": ["iam:AttachRolePolicy", "iam:DetachRolePolicy" ],
        "Resource": "arn:aws:iam::aws:policy/AdministratorAccess"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "ack-iam-controller" {
  role       = aws_iam_role.ack-iam-controller.name
  policy_arn = aws_iam_policy.ack-iam-controller.arn
}

resource "aws_eks_pod_identity_association" "ack-iam-controller" {
  cluster_name    = module.eks.cluster_name
  namespace       = "ack-system"
  service_account = "ack-iam-controller"
  role_arn        = aws_iam_role.ack-iam-controller.arn
}

Step 3. Provision IAM controller chart

IAM-Chart requires aws.region to be set.

export SERVICE=iam
export RELEASE_VERSION=$(curl -sL https://api.github.com/repos/aws-controllers-k8s/${SERVICE}-controller/releases/latest | jq -r '.tag_name | ltrimstr("v")')
export ACK_SYSTEM_NAMESPACE=ack-system
export AWS_REGION=eu-west-1

aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install --create-namespace -n $ACK_SYSTEM_NAMESPACE ack-$SERVICE-controller \
  oci://public.ecr.aws/aws-controllers-k8s/$SERVICE-chart --version=$RELEASE_VERSION --set=aws.region=$AWS_REGION

Step 4. Create sample IAM role

Create example-role and validate role is created in AWS.

apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
  name: example-role
spec:
  name: example-role
  assumeRolePolicyDocument: |
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "ec2.amazonaws.com"
          },
            "Action": [
                "sts:AssumeRole",
                "sts:TagSession"
            ]
        }
      ]
    }
  description: "Example role created by ACK"
  maxSessionDuration: 3600
  policies:
    - arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Step 5. Provisioning Additional controllers

Once IAM controller is installed, you can provision additional controller direct from K8s using ACK

Here is example of S3 controller - Define IAM role with IAM-RSA role for S3 controller - Install S3 controller chart

Create IAM Pod ID role for S3 controller

ack-s3-iam-rsa.yaml

apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
  name: ack-s3-controller
spec:
  name: ack-s3-controller
  assumeRolePolicyDocument: |
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Federated": "arn:aws:iam::<AWS-Account-id>:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/87AD44391B740265AE0AB65382E8D918"
                },
                "Action": "sts:AssumeRoleWithWebIdentity",
                "Condition": {
                    "StringEquals": {
                        "oidc.eks.eu-west-1.amazonaws.com/id/87AD44391B740265AE0AB65382E8D918:sub": "system:serviceaccount:ack-system:ack-s3-controller"
                    }
                }
            }
        ]
    }       
  description: "S3 role created by ACK"
  maxSessionDuration: 3600
  policies:
    - "arn:aws:iam::aws:policy/AmazonS3FullAccess"

Install S3 controller

export SERVICE=s3
export RELEASE_VERSION=$(curl -sL https://api.github.com/repos/aws-controllers-k8s/${SERVICE}-controller/releases/latest | jq -r '.tag_name | ltrimstr("v")')
export ACK_SYSTEM_NAMESPACE=ack-system
export AWS_REGION=eu-west-1

aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install --create-namespace -n $ACK_SYSTEM_NAMESPACE ack-$SERVICE-controller \
  oci://public.ecr.aws/aws-controllers-k8s/$SERVICE-chart --version=$RELEASE_VERSION --set=aws.region=$AWS_REGION

Provision Sample bucket

apiVersion: s3.services.k8s.aws/v1alpha1
kind: Bucket
metadata:
  name: example-ack-bucket-sample
spec:
  name: example-ack-bucket-051020251400
  versioning:
    status: Enabled
  tagging:
    tagSet:
      - key: environment
        value: dev
      - key: owner
        value: "Denny Vettom"

Step 6. Provision sample app with S3 access

Scenario: Deploy app that require access to S3 bucket. Configure bucket and IAM-RSA permissions using ACK

Create necessary access policy using ACK
Create Role with IAM-RSA and attach policy
Deploy app with configured SA and validate

ack-demo-iam-rsa.yaml

apiVersion: iam.services.k8s.aws/v1alpha1
kind: Policy
metadata:
  name: demo-app-bucket-access-policy
spec:
  name: demo-app-bucket-access-policy
  description: "Allows ACK EKS Controller to manage Pod Identity Associations"
  policyDocument: |
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [ "s3:GetObject", "s3:ListBucket", "s3:PutObject"],
          "Resource": [ "arn:aws:s3:::eks-auto-demo-bucket-102025",  "arn:aws:s3:::eks-auto-demo-bucket-102025/*"]
        }
      ]
    }
---
apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
  name: ack-demo-app-role
spec:
  name: ack-demo-app-role
  assumeRolePolicyDocument: |
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Federated": "arn:aws:iam::<AWS-Account-id>:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/87AD44391B740265AE0AB65382E8D918"
                },
                "Action": "sts:AssumeRoleWithWebIdentity",
                "Condition": {
                    "StringEquals": {
                        "oidc.eks.eu-west-1.amazonaws.com/id/87AD44391B740265AE0AB65382E8D918:sub": "system:serviceaccount:default:ack-demo-app-sa"
                    }
                }
            }
        ]
    }       
  description: "Example role created by ACK"
  maxSessionDuration: 3600
  policies:
    - arn:aws:iam::<AWS-Account-id>:policy/demo-app-bucket-access-policy

Apply the configuration and verify. kubectl apply -f ack-demo-iam-rsa.yaml

Deploy Demo app

This is a pod with AWS CLI installed. No credentials configured, it will use authentication provided by IAM-RSA

deploy-ack-demo-app.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ack-demo-app-sa
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ack-demo-app
  name: ack-demo-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ack-demo-app
  strategy: {}
  template:
    metadata:
      labels:
        app: ack-demo-app
    spec:
      serviceAccountName: ack-demo-app-sa
      containers:
      - image: dennysv/dvawscli
        name: dvawscli
        resources: {}

Verify bucket access

SSH on to the pod

$ aws s3 ls eks-auto-demo-bucket-102025/
$ aws s3 cp /etc/os-release  s3://eks-auto-demo-bucket-102025/
$ aws s3 ls eks-auto-demo-bucket-102025/
$ aws s3 crm os-release  s3://eks-auto-demo-bucket-102025/os-release

You have successfully configured ACK and deployed sample App making use of ACK.