Opscraft
IP addressing & subnetting · Lesson 7 of 9 · 6 min read

IP planning in AWS and EKS

AWS rules

  • VPC CIDR: from /16 (65,536) down to /28 (16). You can add secondary CIDRs later (up to 5 by default) — including from 100.64.0.0/10.
  • Subnet: /16 to /28, inside the VPC CIDR, in one AZ.
  • AWS reserves 5 addresses in every subnet — for 10.0.1.0/24:
    • .0 network address
    • .1 VPC router
    • .2 DNS resolver (the VPC's base + 2 is also the Route 53 Resolver; e.g. 10.0.0.2 for 10.0.0.0/16)
    • .3 reserved for future use
    • .255 broadcast (not supported, but reserved)
  • So a /28 has 11 usable, a /24 has 251, a /20 has 4,091.
  • CIDRs cannot overlap for peering or Transit Gateway routing, and shouldn't overlap with on-prem ranges reachable over VPN/Direct Connect.
Enlarge diagram.0network.1VPC router.2DNS resolver.3reserved.4 … .254yours: 251.255broadcastSubnet 10.0.1.0/24
Figure 1. AWS keeps five addresses in every subnet, so a /24 has 251 usable addresses and a /28 only 11.
Swipe sideways to see all of it, or tap to open it full screen.

Planning for many accounts

A typical plan for an organisation with multiple accounts and regions:

Enlarge diagram10.0.0.0/8the company space10.16.0.0/12ap-south-1 (Mumbai)10.16.0.0/16prod network10.17.0.0/16prod apps10.18.0.0/16staging10.19.0.0/16dev10.32.0.0/12ap-southeast-110.200.0.0/16reserved for on-premises and VPN100.64.0.0/10EKS pod secondary CIDRs (never routed between VPCs)
Figure 2. An IP plan for an organisation: aligned blocks per region and environment, so nothing overlaps and routes summarise.
Swipe sideways to see all of it, or tap to open it full screen.
  • Allocate aligned blocks per region/environment so routes summarise.
  • Leave room to grow; re-IPing later is painful.
  • Track allocations centrally — AWS VPC IPAM allocates non-overlapping CIDRs from pools per region/OU, monitors usage and enforces rules (or a spreadsheet/NetBox in smaller orgs).
  • Why 10.x for corporate plans: the largest private space (16.7M addresses) to carve non-overlapping ranges for every account and environment.

EKS: pods eat IPs

With the Amazon VPC CNI, each pod gets a real VPC IP from the node's subnet. Sizing consequences:

  • A node can hold IPs only via its ENIs. Classic formula for max pods per node: ENIs × (IPv4 addresses per ENI − 1) + 2. Example: m5.large supports 3 ENIs × 10 IPs → 3 × (10 − 1) + 2 = 29 pods.
  • The CNI keeps warm IPs/ENIs pre-allocated per node, consuming addresses even before pods exist (tunable: WARM_IP_TARGET, MINIMUM_IP_TARGET).
  • A cluster of 50 nodes × 30 pods needs ~1,500+ IPs plus warm pools — a /24 per AZ is far too small. Size private subnets for pods: /19–/20 per AZ is common.
Enlarge diagramM5.LARGE NODEENI 1ENIpodpodpodpodpodpodpodpodpodENI 2ENIpodpodpodpodpodpodpodpodpodENI 3ENIpodpodpodpodpodpodpodpodpod3 × (10 − 1) = 27 pod IPs, + 2 pods that use the host network = 29
Figure 3. Why an m5.large runs at most 29 pods with the VPC CNI: 3 network interfaces × (10 IPs − 1 for the interface itself) + 2 host-network pods.
Swipe sideways to see all of it, or tap to open it full screen.

When IPs run out (pods stuck Pending/ContainerCreating with "failed to assign an IP address"):

  1. Prefix delegation — assign /28 prefixes (16 IPs) to ENI slots instead of single IPs, raising pods per node dramatically (e.g. m5.large up to 110), and allocating IPs in chunks. Needs contiguous free /28 blocks — subnet fragmentation can bite; subnet CIDR reservations help.
  2. Secondary CIDR + custom networking — add 100.64.0.0/16 (non-routable outside the VPC) to the VPC, create pod subnets from it, and configure ENIConfig per AZ so pods use those subnets while nodes stay in the original ones.
  3. New, larger subnets for new node groups.
  4. IPv6 clusters for the long term.
  5. Release unused ENIs and tune warm pool settings.
Enlarge diagramSecondary IPs (default)Prefix delegationOne IP per slotm5.large: about 29 podsIPs allocated one at a timeA /28 (16 IPs) per slotm5.large: up to 110 podsNeeds free, contiguous /28 blocks in the subnet
Figure 4. Prefix delegation: each interface slot holds a /28 block of 16 addresses instead of one, so the same node can run many more pods.
Swipe sideways to see all of it, or tap to open it full screen.

Kubernetes CIDRs

  • Pod CIDR — on EKS with the VPC CNI, pods use VPC subnets; with overlay CNIs, a separate cluster pod range.
  • Service CIDR — virtual ClusterIPs (EKS default 10.100.0.0/16 or 172.20.0.0/16 depending on the VPC range; configurable at creation). It's never routed outside the cluster but must not overlap with the VPC, pods, or networks pods need to reach — otherwise traffic to those addresses is hijacked by kube-proxy rules.