Opscraft
Docker Swarm · Lesson 4 of 10 · 3 min read

Networking and the routing mesh

Overlay networks

An overlay network spans all nodes using VXLAN encapsulation over UDP 4789. Services attached to the same overlay can reach each other by service name through Swarm's embedded DNS.

Enlarge diagramNODE 1 · 10.0.1.11NODE 2 · 10.0.1.12api10.0.9.5VXLANencapsulatesdb10.0.9.6VXLANencapsulatesUDP 4789 between hostsData is not encrypted by default: add --opt encrypted for IPsec between nodes.
Figure 1. An overlay network spans hosts: containers on different nodes talk as if on one LAN, with traffic wrapped in VXLAN over UDP 4789.
Swipe sideways to see all of it, or tap to open it full screen.
docker network create -d overlay --attachable backend
docker network create -d overlay --opt encrypted secure-backend
  • Management/control traffic is always encrypted (mTLS). Data traffic on overlays isn't encrypted by default — --opt encrypted adds IPsec between nodes (with some CPU overhead).
  • --attachable lets standalone docker run containers join the overlay (handy for debugging).
  • Each node also has a docker_gwbridge bridge that connects overlay containers to the outside world.

Service discovery: VIP vs DNS round robin

By default each service gets a virtual IP (VIP). DNS for web returns the VIP, and the kernel's IPVS load-balances connections across healthy tasks. Alternatively --endpoint-mode dnsrr returns the task IPs directly (DNS round robin) — needed for some clients and when using your own load balancer, but DNS caching makes balancing less even. tasks.web always resolves to the individual task IPs.

The ingress routing mesh

Publishing a port (-p 80:80 / ports: in a stack) uses the special ingress overlay network:

  • The port is opened on every node in the swarm, whether or not it runs a task of that service.
  • A connection to any node's port is load-balanced (IPVS) to a healthy task anywhere in the cluster.

So an external load balancer (an AWS NLB/ALB) can target all nodes on that port, and tasks can move freely.

Enlarge diagramAWS NLBtargets all nodes:80node 1 :80 openruns a taskweb tasknode 2 :80 openno task herenode 3 :80 openruns a taskweb taskIPVS forwardsTrade-off: an extra hop and the client IP is lost. Host-mode publishing avoids both.
Figure 2. The ingress routing mesh: a published port opens on every node, and any node forwards the connection to a healthy task wherever it runs.
Swipe sideways to see all of it, or tap to open it full screen.

Trade-offs: an extra network hop, and the client's source IP is lost (it's SNATed). If you need the source IP or minimal latency, publish in host mode:

docker service create --name edge --mode global \
  --publish mode=host,target=80,published=80 nginx

Host mode binds only on nodes running a task, and only one task per node can use that port — so it pairs naturally with global mode. A typical design: Traefik or Nginx as a global service in host mode on edge nodes, routing to internal services by name.