Skip to content

Alertmanager Email Notifications with Gmail SMTP

Alertmanager

Prometheus Alertmanager supports dispatching alert notifications via SMTP email servers, including Google Workspace / Gmail.

This guide demonstrates how to configure email alerting securely in Kubernetes using kube-prometheus-stack with secrets mounted as files.

Step 1. Generate Gmail App Password

  1. In your Google Account, navigate to Security → 2-Step Verification → App passwords.
  2. Generate a dedicated App password for Alertmanager (16-character token).

Step 2. Store SMTP Credentials in Kubernetes Secret

smtp-secret.yaml 

apiVersion: v1
kind: Secret
metadata:
  name: alertmanager-smtp-secret
  namespace: monitoring
type: Opaque
stringData:
  gmail-password: "your-16-char-app-password"


kubectl apply -f smtp-secret.yaml

Step 3. Configure kube-prometheus-stack Alertmanager Values

alertmanager-email-values.yaml 

alertmanager:
  enabled: true
  config:
    global:
      smtp_smarthost: "smtp.gmail.com:587"
      smtp_from: "alerts@vettom.online"
      smtp_auth_username: "vettom@gmail.com"
      smtp_auth_password_file: "/etc/alertmanager/secrets/gmail-password"
      smtp_require_tls: true
    receivers:
      - name: "null"
      - name: "email-alerts"
        email_configs:
          - to: "oncall@vettom.online"
            send_resolved: true
    route:
      receiver: "email-alerts"
      group_by: ["alertname", "cluster", "severity"]
      group_wait: 30s
      group_interval: 5m
      repeat_interval: 4h
  alertmanagerSpec:
    volumes:
      - name: smtp-secret-volume
        secret:
          secretName: alertmanager-smtp-secret
    volumeMounts:
      - name: smtp-secret-volume
        mountPath: /etc/alertmanager/secrets
        readOnly: true

Step 4. Reload Alertmanager Configuration

Prometheus Operator includes a config-reloader container that detects secret and ConfigMap updates automatically. To trigger an immediate reload:

kubectl exec -it -n monitoring alertmanager-kube-prometheus-stack-alertmanager-0 -c alertmanager \
  -- curl -X POST http://localhost:9093/-/reload