Amazon EKS · Lesson 10 of 11 · 3 min read
Troubleshooting EKS-specific problems
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
| Symptom | Likely cause / fix |
|---|---|
| Nodes fail to join ("Instances failed to join the cluster") | Node IAM role missing access entry (or aws-auth mapping); subnets can't reach the API endpoint (private endpoint + no route, security groups); missing VPC endpoints for STS/ECR/EC2 in private-only clusters; wrong AMI/bootstrap user data; check /var/log/cloud-init-output.log and kubelet logs on the instance |
Pods stuck ContainerCreating: failed to assign an IP address |
Subnet IP exhaustion — prefix delegation, secondary CIDR, bigger subnets |
| ImagePullBackOff from ECR in private subnets | No NAT and no ECR api/dkr + S3 gateway endpoints; node role missing ECR read policy; cross-account repository policy |
| ALB not created for an Ingress | LB controller logs; IngressClass alb; subnet tags; IAM permissions; annotation typos |
| ALB targets unhealthy | Health check path/port, readiness, security groups between ALB and pods/nodes |
Pod gets AccessDenied calling AWS |
ServiceAccount annotation/association missing, trust policy sub doesn't match system:serviceaccount:<ns>:<sa>, pod not using that ServiceAccount, SDK too old for web identity/Pod Identity, permissions policy too narrow |
| Pods using the node role instead of IRSA | Credential chain falling back to IMDS — IRSA env vars missing (pod created before annotation → restart pods) |
kubectl Unauthorized / You must be logged in |
Wrong AWS profile/role, no access entry for the principal, expired SSO session; aws sts get-caller-identity |
| DNS timeouts | CoreDNS scaling, ndots, VPC resolver per-ENI limits → NodeLocal DNSCache |
| Karpenter not launching nodes | NodePool limits, requirements too narrow, EC2NodeClass discovery tags, IAM, Spot capacity — kubectl get nodeclaims, controller logs |
| Upgrade broke workloads | Removed API versions in manifests/charts, add-on incompatibility |