Node components and the pod lifecycle
kubelet
The agent on every node. It:
- Registers the node and sends heartbeats (Lease objects in
kube-node-leaseplus NodeStatus updates). - Watches for pods bound to its node, and drives the container runtime via the CRI (Container Runtime Interface) to create the pod sandbox, pull images, and start containers.
- Mounts volumes (with CSI drivers), sets up the network via the CNI plugin.
- Runs probes (liveness, readiness, startup) and restarts containers per
restartPolicy. - Reports pod status back to the API server.
- Enforces resource limits via cgroups, and performs node-pressure eviction when memory, disk (nodefs/imagefs) or PIDs run low (e.g.
memory.available<100Mi). - Runs static pods from manifests in
/etc/kubernetes/manifests(that's how kubeadm runs the control plane itself), visible in the API as mirror pods. - Garbage-collects dead containers and unused images.
The kubelet's PLEG (pod lifecycle event generator) relists containers periodically; "PLEG is not healthy" means the runtime is too slow to respond — often overloaded nodes or a stuck container runtime — and leads to NotReady.
Container runtime
Each pod first gets a sandbox (the "pause" container) that holds the pod's network namespace; app containers join it — that's why containers in a pod share an IP and can talk over localhost. Debug the runtime directly with crictl ps, crictl logs, crictl inspectp, crictl images.
kube-proxy
Implements Services on each node by programming packet rules from EndpointSlices:
- iptables mode (default): NAT rules — a Service's ClusterIP:port DNATs to a randomly chosen backend pod (probability rules). Rule count grows with Services × endpoints; updates get slower at large scale.
- IPVS mode: kernel load balancer with hash tables — scales better and offers algorithms (rr, least-connection, source hash).
- nftables mode (newer releases) as the iptables successor.
- Replaced entirely by eBPF in Cilium's kube-proxy-free mode.
kube-proxy doesn't proxy traffic itself in these modes — it only writes kernel rules.
CNI plugin
Gives every pod an IP and connectivity. The Kubernetes network model requires: every pod gets its own IP; pods can reach all other pods without NAT; nodes can reach all pods. How it's implemented varies:
| CNI | Approach |
|---|---|
| Amazon VPC CNI | Pods get real VPC IPs from ENIs — no overlay (EKS default) |
| Calico | BGP routing or VXLAN/IP-in-IP overlay; strong NetworkPolicy support |
| Cilium | eBPF dataplane, kube-proxy replacement, L7 policies, observability (Hubble) |
| Flannel | Simple VXLAN overlay, no NetworkPolicy |
What happens when you create a Deployment
kubectl applysends the Deployment to the API server: authn → authz → admission → validation → etcd.- The Deployment controller (watching Deployments) creates a ReplicaSet for the pod template (named with a hash of the template).
- The ReplicaSet controller creates N Pod objects (no node yet).
- The scheduler filters and scores nodes, then binds each pod to a node.
- The kubelet on that node sees the pod: pulls images (via containerd), sets up the sandbox and networking (CNI assigns an IP), mounts volumes, runs init containers in order, then starts app containers.
- Probes run; when readiness passes, the pod condition
Ready=True. - The EndpointSlice controller adds the pod IP to matching Services' EndpointSlices; kube-proxy on every node updates rules; ingress/ALB controllers register targets; CoreDNS answers for the Service.
Pod phases, conditions and container states
- Phases:
Pending(accepted, not yet running — scheduling, image pulls),Running(bound, at least one container running),Succeeded,Failed,Unknown(node unreachable). - Conditions:
PodScheduled,Initialized,ContainersReady,Ready(and readiness gates). - Container states:
Waiting(reasons: ContainerCreating, ImagePullBackOff, CrashLoopBackOff, CreateContainerConfigError),Running,Terminated(reason, exit code). - restartPolicy:
Always(Deployments),OnFailureorNever(Jobs). Restarts back off exponentially up to 5 minutes — that's CrashLoopBackOff.
Init containers and sidecars
- Init containers run to completion, in order, before app containers start: wait for a dependency, run migrations, fetch config, fix volume permissions. If one fails, the pod restarts them per policy.
- Sidecars run alongside the app: log shippers, proxies (Envoy), secret refreshers. Since Kubernetes 1.28 (stable in 1.33) there are native sidecars: an init container with
restartPolicy: Alwaysstarts before app containers, keeps running, and is stopped after them — fixing old problems like Jobs never completing because a sidecar kept running.
Pod termination
On deletion: the pod gets a deletion timestamp and is removed from endpoints; the kubelet runs preStop hooks, sends SIGTERM, waits up to terminationGracePeriodSeconds (default 30), then SIGKILL. Endpoint removal and SIGTERM happen concurrently, so apps need a short preStop delay to avoid dropped requests (covered in the deployment-strategies topic).