A troubleshooting method
This topic is Kubernetes on the job: a pod won't start, a Service returns 503, a node goes NotReady, a deploy is stuck, a namespace won't delete. Knowing the commands isn't enough; senior interviews want a systematic method, the ability to decode every common status and error message quickly, and incident stories told well. Everything here is organised by symptom, with the exact commands and the usual causes in order of likelihood, plus debugging for Flux-based GitOps setups.
Work from symptom to layer
Most Kubernetes problems live in one of these layers:
Ask in order: What changed? (deploys, config, node/cluster upgrades, traffic) What's the scope? (one pod, one node, one namespace, everything) Where does the request stop?
The core loop
kubectl get pods -n shop -o wide # status, restarts, node, IP
kubectl describe pod orders-7d9f-abc -n shop # spec, state, last state, exit code, EVENTS
kubectl logs orders-7d9f-abc -n shop -c orders # current container logs
kubectl logs orders-7d9f-abc -n shop --previous # the crashed instance's logs
kubectl get events -n shop --sort-by=.lastTimestamp # recent namespace events
kubectl get events -A --field-selector type=Warning # warnings cluster-wide
describe + --previous logs solve the majority of pod problems. Events are kept only about one hour by default, so look early (or ship events to your logging system with an event exporter).
More tools
kubectl get pod orders-7d9f-abc -o yaml # full spec and status (conditions, containerStatuses)
kubectl get deploy,rs,pods -l app=orders -n shop # whole ownership chain
kubectl rollout status deploy/orders -n shop
kubectl top pods -n shop --containers # usage (needs metrics-server)
kubectl top nodes
kubectl exec -it orders-7d9f-abc -n shop -- sh # shell (if the image has one)
kubectl port-forward svc/orders 8080:80 -n shop # test a Service from your laptop
kubectl auth can-i get secrets --as=system:serviceaccount:shop:orders -n shop
kubectl get --raw /readyz?verbose # API server health
Debugging minimal images and nodes
Distroless images have no shell. Use ephemeral debug containers:
kubectl debug -it orders-7d9f-abc -n shop --image=nicolaka/netshoot --target=orders
# shares the target container's process namespace: ps, curl, dig, tcpdump, ss
kubectl debug orders-7d9f-abc -n shop -it --copy-to=orders-debug --container=orders -- sh # copy of the pod with changes
kubectl debug node/ip-10-20-1-15.ap-south-1.compute.internal -it --image=ubuntu
# host filesystem at /host: chroot /host; journalctl -u kubelet; crictl ps
A throwaway network test pod is invaluable:
kubectl run -it --rm netshoot -n shop --image=nicolaka/netshoot -- bash
# dig, curl, nc, mtr, tcpdump from inside the cluster network