Skip to content

Karpenter deploy specific node type

karpenter EKS Auto mode comes with pre-installed Karpenter Autoscaler, default NodeClass and a default Nodepool. Default Nodepool is configured to provision AMD architecture, on-demand servers.

Some applications may require specific type of node to run workloads like GPU, or ARM architecture based servers, or you may want to save cost by running stateless applications on spot instances. Karpenter can be configured to support these requirements and by combining with kubernetes nodeAffinity, taints and tolerations you can provision and run workloads in specific node types.

Default NodeClass is used here, to customize further, define own NodeClass

Schedule workload on specific type of server

In this example, I am creating a nodepool that will provision Memory Optimized Spot instances. Taints are applied to the nodepool so that only pods with toleration can run on this node. Taints are not always enough to run specific application on specific Node, so I am also adding node label so that pods and select specific node using nodeaffinity.

Custom Nodepool

In below example I am configuring a nodepool that will provision Memory Optimized M Spot instance only. It has a Node label instancegroup: spot-m-instancegroup and taint applied. Taints are applied so that only pods with specific toleraton can be scheduled on these nodes.

spot-m-nodepool.yaml 

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-m-nodepool
spec:
  template:
    metadata: 
      labels:
        instancegroup: spot-m-instancegroup
    spec:
      expireAfter: 336h
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
      requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: ["spot"]
      - key: eks.amazonaws.com/instance-category
        operator: In
        values: ["m"]
      taints:
        - key: instance-type
          value: "spot-m"
          effect: NoSchedule

Application with NodeSelect and Toleration

Below is a sample application that has nodeSelector and tolerations so that it can only be provisioned on nodes with nodeLabel instancegroup: spot-m-instancegroup and it has necessary toleration for Pods. This will trigger Karpenter nodePool spot-m-nodepool to provision a new node. spot-m-echoserver.yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echoserver
  namespace: default
spec:
  selector:
    matchLabels:
      app: echoserver
  replicas: 1
  template:
    metadata:
      labels:
        app: echoserver
    spec:
      containers:
        - image: k8s.gcr.io/e2e-test-images/echoserver:2.5
          name: demo-echo
          ports:
            - containerPort: 8080
      nodeSelector:
        instancegroup: spot-m-instancegroup
      tolerations:
        - key: "instance-type"
          operator: "Equal"
          value: "spot-m"
          effect: "NoSchedule"

If you describe pod, you will see that nodepool spot-m-nodepool is selected and application gets deployed on a Spot instance with M series server.