Control plane components in depth
kube-apiserver
The only component that talks to etcd, and the front door for everyone: kubectl, controllers, the scheduler, kubelets. It's stateless and horizontally scalable behind a load balancer.
Every request goes through a pipeline:
- Authentication — who are you? Client certificates, bearer tokens (ServiceAccount JWTs), OIDC tokens, webhook token authenticators (EKS uses the AWS IAM authenticator / access entries). Failure → 401.
- Authorization — may you do this verb on this resource? Usually RBAC (plus Node authorizer for kubelets, webhooks). Failure → 403.
- Mutating admission — may modify the object: defaulting, injecting sidecars (Istio), adding labels, the ServiceAccount admission plugin,
MutatingAdmissionWebhooks (Gatekeeper mutation, Kyverno). - Schema validation of the (mutated) object.
- Validating admission — accept/reject only:
ValidatingAdmissionWebhook(Gatekeeper/Kyverno),ValidatingAdmissionPolicy(CEL, built-in), PodSecurity, ResourceQuota, LimitRanger. - Persist to etcd; notify watchers.
Other features: API aggregation (metrics-server's metrics.k8s.io is an aggregated API), API Priority and Fairness (protects the server from noisy clients), audit logging, server-side apply with field ownership.
etcd
A distributed, strongly consistent key-value store holding all cluster state (/registry/... keys). It uses the Raft consensus algorithm:
- One leader handles writes and replicates log entries to followers; an entry is committed once a majority acknowledges it.
- Quorum = ⌊N/2⌋+1 — run 3 or 5 members (odd), spread across failure domains. 3 tolerates 1 failure, 5 tolerates 2.
- Lose quorum → the cluster becomes read-only/unavailable for writes: no new pods, no scaling, no changes. Running containers keep running (kubelets keep what they have) but nothing can be managed.
Performance depends on disk fsync latency (use fast SSD; on cloud, provisioned IOPS) and low network latency between members. Symptoms of a slow etcd: high API latency, leader elections, "etcdserver: request timed out". The default database size quota is 2 GB (often raised to 8 GB); old revisions must be compacted and the DB defragmented or it hits the quota and goes read-only ("mvcc: database space exceeded").
Backup and restore (self-managed clusters):
ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-$(date +%F).db \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key
etcdutl snapshot status /backup/etcd-2026-10-25.db
etcdutl snapshot restore /backup/etcd-2026-10-25.db --data-dir /var/lib/etcd-restored
# then point the etcd static pod manifest at the new data dir
Losing etcd with no backup means losing every object and Secret — you'd rebuild the cluster from Git (one more reason for GitOps). Secrets are stored unencrypted (base64) unless you configure encryption at rest with an EncryptionConfiguration (aescbc/aesgcm/KMS provider). On EKS, AWS runs etcd for you and you enable KMS envelope encryption for Secrets.
kube-scheduler
Watches for pods with no spec.nodeName and assigns each to a node. For each pod it runs the scheduling framework:
- Filtering (predicates) — remove nodes that can't run the pod: insufficient allocatable CPU/memory for the pod's requests, node selector/affinity mismatch, untolerated taints, port conflicts, volume topology (EBS in another AZ), max pods per node, pod (anti-)affinity violations.
- Scoring (priorities) — rank the remaining nodes: least/most allocated (spreading vs bin-packing), balanced resource use, image locality (image already present), topology spread, preferred affinities.
- Binding — writes the chosen node into the pod (a Binding object). The kubelet on that node takes over.
If no node passes filtering, the pod stays Pending with an event like 0/6 nodes are available: 3 Insufficient memory, 3 node(s) had untolerated taint {dedicated: gpu}. Preemption may evict lower-priority pods to make room for a higher-priority one. Cluster Autoscaler/Karpenter watch for unschedulable pods and add nodes.
The scheduler only considers requests, never actual usage — a node can be "full" while nearly idle, or overloaded while "half empty" if requests are unrealistic.
kube-controller-manager
One binary running many controllers, each a reconciliation loop: Deployment, ReplicaSet, StatefulSet, DaemonSet, Job, CronJob, Namespace (cleanup), Node (heartbeat monitoring, taints for unreachable nodes), EndpointSlice, ServiceAccount and token, PersistentVolume binder and expansion, garbage collector, HPA, ResourceQuota, certificate signing, TTL-after-finished. Controllers use leader election so only one active instance runs in an HA control plane.
cloud-controller-manager
Cloud-specific loops split out of core Kubernetes: the node controller (initialising nodes with cloud metadata, deleting Node objects when instances are terminated), the route controller, and the service controller (creating cloud load balancers for type: LoadBalancer). On EKS, the AWS Load Balancer Controller (an add-on you install) handles ALBs/NLBs with much more capability.
High availability
A production control plane runs 3+ API servers behind a load balancer, 3 or 5 etcd members (stacked on control-plane nodes or external), and scheduler/controller-manager replicas with leader election. EKS does all of this for you across multiple AZs.