Gateway API (Envoy) with EKS Network Load Balancer

Gateway API is the next generation of Kubernetes Ingress, Load Balancing, and Service Mesh APIs. This document focuses on North/South ingress traffic using Envoy Gateway.

Gateway API resources

• GatewayClass — Defines the type of Gateway (internal/external), cloud-provider-specific settings, and implementation software
• Gateway — Creates a load balancer and listeners. Each Gateway creates a new load balancer.
• HTTPRoute — Defines routing rules for traffic: backend targets, redirects, canary splits, traffic mirroring, etc.

Implement Envoy Gateway

Setup

Create an EKS cluster with VPC-CNI and the AWS Load Balancer controller installed. Complete Terraform code is available at EKS-Envoy-Gateway in the aws-eks-terraform repo.

Install Envoy Gateway

helm install gateway oci://docker.io/envoyproxy/gateway-helm --version v1.0.2 -n gateway --create-namespace

Create an EnvoyProxy to configure the NLB

Envoy Gateway spins up Envoy Proxy instances per Gateway. Applying an EnvoyProxy configuration lets you control its behaviour and define annotations required by the AWS Load Balancer controller. The configuration below creates an internet-facing Network Load Balancer with target type ip — this requires both the AWS Load Balancer controller and VPC-CNI.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: external-proxy-config
  namespace: gateway
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
          service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
          service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: preserve_client_ip.enabled=true

Create a GatewayClass and configure it to use the EnvoyProxy

An Envoy GatewayClass can be linked to an EnvoyProxy configuration. Any Gateway created using this GatewayClass inherits that configuration.

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: external-gatewayclass
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: external-proxy-config
    namespace: gateway

Create a Gateway and listeners

Creating a Gateway provisions an NLB and an Envoy Proxy pod to handle requests. Define the required listeners below. If using Cert-Manager for certificate management, add the necessary annotation. For HTTPS listeners you must specify a hostname and certificate reference.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: external-gateway
  namespace: gateway
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-DNS
spec:
  gatewayClassName: external-gatewayclass
  listeners:
    - name: http
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: All
    # - name: https
    #   hostname: "mygateway.vettom.pages.dev"
    #   port: 443
    #   protocol: HTTPS
    #   allowedRoutes:
    #     namespaces:
    #       from: All
    #   tls:
    #     certificateRefs:
    #       - kind: Secret
    #         group: ""
    #         name: "mygateway-cert"
    #         namespace: gateway

Add an HTTPRoute for your application

Define an HTTPRoute to specify how traffic is routed to your application. In the parentRefs section, specify which Gateway to attach to. See the HTTPRoute reference for all available options.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: dvenvoy-http
  namespace: echoserver
spec:
  parentRefs:
    - name: external-gateway
      sectionName: http
      namespace: gateway
  hostnames:
    - mygateway.vettom.pages.dev
    - mygateway1.vettom.pages.dev
  rules:
    - backendRefs:
        - kind: Service
          name: echoserver
          port: 80
      matches:
        - path:
            type: PathPrefix
            value: /