Opscraft
Kubernetes, EKS, GitOps & releases

Kubernetes troubleshooting

11 lessons · 42 min · 3 tests ·

This topic is Kubernetes on the job: a pod won't start, a Service returns 503, a node goes NotReady, a deploy is stuck, a namespace won't delete. Knowing the commands isn't enough; senior interviews want a systematic method, the ability to decode every common status and error message quickly, and incident stories told well. Everything here is organised by symptom, with the exact commands and the usual causes in order of likelihood, plus debugging for Flux-based GitOps setups.

Stage 3 · Production platforms

Know these first

After this topic you can

  • Diagnose CrashLoopBackOff, Pending pods and failed rollouts
  • Debug Service, DNS and ingress problems
  • Work through a node or cluster incident methodically

Lessons

  1. 1A troubleshooting method 4 min read
    • Work from symptom to layer
    • The core loop
    • More tools
    • Debugging minimal images and nodes
  2. 2Pods that won't start 6 min read
    • Pending — not scheduled
    • ContainerCreating — scheduled, not started
    • Image pull errors
    • CreateContainerConfigError
    • Init containers
  3. 3Pods that crash or restart 5 min read
    • CrashLoopBackOff
    • Exit codes
    • OOMKilled
    • Liveness probes killing healthy apps
    • Other crash causes
  4. 4Probes, readiness and traffic 3 min read
    • Running but not Ready
    • Probe design summary
    • Rollouts that get stuck
  5. 5Resources and performance 3 min read
    • CPU throttling
    • Evictions
    • HPA not scaling
    • Noisy neighbours
  6. 6Networking: Services, DNS, policies, ingress 5 min read
    • Service has no endpoints
    • Testing connectivity step by step
    • DNS failures
    • NetworkPolicies
    • Ingress problems
    • Pods can't reach AWS services or the internet
  7. 7Storage problems 3 min read
    • PVC stuck Pending
    • Multi-Attach and AZ issues
    • Volume permissions and capacity
  8. 8Node problems 3 min read
    • A node is NotReady
    • Drains that never finish
  9. 9Cluster-level problems 3 min read
    • Things broke after a cluster upgrade
    • Namespace stuck in Terminating
    • Admission webhooks blocking everything
    • RBAC "forbidden"
    • Certificates and control plane
  10. 10GitOps (Flux) troubleshooting 3 min read
  11. 11Incident stories and interview answers 4 min read
    • How to tell a Kubernetes incident story
    • "A pod is in CrashLoopBackOff. Walk me through it."
    • "A pod is stuck in Pending."
    • "Production pods are healthy but users see errors."
    • "Pods restart every few hours with OOMKilled."
    • "A node is NotReady."
    • "Someone kubectl-edited production. What happens with Flux?"
    • "How do you prevent noisy neighbours and make apps highly available?"