Skip to content

ingress

Complete Amazon EKS Cluster Build with Terraform & ALB

EKS Architecture

Creating a functional Amazon EKS cluster from scratch requires configuring VPC subnet discovery tags, managed node groups, VPC CNI prefix delegation, and the AWS Load Balancer Controller.

📁 Source Code: aws-eks-terraform / EKS-Cluster-ALB

📺 Video Walkthrough

What's Included

  • VPC: 2 private subnets for worker nodes and 2 public subnets with ELB discovery tags.
  • EKS Cluster: Kubernetes 1.31+ with API-only access authentication (EKS Access Entries).
  • Managed Node Group: Single Spot instance running optimized Bottlerocket OS.
  • VPC-CNI Add-on: Enabled with prefix delegation for high pod density.
  • AWS Load Balancer Controller: Deployed via Helm to provision ALBs in ip target mode.

EKS: Avoiding 502/504 Errors & Timeouts During Rolling Deployments (ALB)

Target Group Status

When exposing Amazon EKS applications through an AWS Application Load Balancer (ALB) using IP target mode, rolling updates can sometimes trigger brief 502 Bad Gateway / 504 Gateway Timeout errors.

Root Causes

  1. Slow Target Registration: Kubernetes marks a new pod as Ready before the AWS ALB Target Group finishes health checking and registering the pod's IP address.
  2. Abrupt Termination: Kubernetes sends SIGTERM and terminates an old pod before the ALB finishes draining active connections from the target group.

The Two-Part Solution

1. Enable Pod Readiness Gates

Configuring Pod Readiness Gates prevents Kubernetes from terminating old pods until the new pod's IP is fully registered and healthy inside the AWS ALB target group.

Enable readiness gate injection by labeling your application namespace:

kubectl label namespace default elbv2.k8s.aws/pod-readiness-gate-inject=enabled

2. Configure preStop Sleep Hook & Deregistration Delay

Add a preStop hook to delay pod shutdown while the ALB deregisters the target, and tune the Target Group deregistration delay:

deployment.yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  template:
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: app
          image: nginx:latest
          lifecycle:
            preStop:
              exec:
                command: ["/bin/sh", "-c", "sleep 30"]

ingress.yaml 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=30
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-service
                port:
                  number: 80