Opscraft
Docker Swarm · Lesson 3 of 10 · 5 min read

Services, tasks and scheduling

The desired-state model

You declare a service — image, replicas, ports, networks, update policy, constraints. The managers' orchestrator compares desired vs actual state continuously and creates or removes tasks. A task is one container slot; it's assigned to a node, and the node's agent runs it. If a task fails or its node dies, the orchestrator creates a replacement task elsewhere — tasks are never "restarted on another node", they're replaced.

Enlarge diagramdesiredweb: 3 replicasactual2 running (1 nodedied)orchestratorschedules 1 new taskcomparedifferencerepeat forever
Figure 1. Swarm keeps comparing what you asked for with what is running, and fixes the difference: a dead task is replaced automatically.
Swipe sideways to see all of it, or tap to open it full screen.

Task states progress: NEW → PENDING → ASSIGNED → ACCEPTED → PREPARING → STARTING → RUNNING, ending in COMPLETE, FAILED, SHUTDOWN, REJECTED or ORPHANED.

docker service create --name web --replicas 3 -p 80:80 nginx:1.27
docker service ls                         # REPLICAS 3/3
docker service ps web                     # tasks, nodes, states, errors
docker service ps --no-trunc web          # full error messages
docker service inspect --pretty web
docker service logs -f web                # aggregated logs from all tasks
docker service scale web=5
docker service rm web

Replicated vs global

  • Replicated (default): run exactly N tasks anywhere suitable.
  • Global: run one task on every node (that matches constraints) — new nodes automatically get one. Use for monitoring agents (node-exporter, cAdvisor), log shippers, security agents. Same idea as a Kubernetes DaemonSet.

There are also replicated-job and global-job modes for run-to-completion tasks.

Enlarge diagramREPLICATED · 3 REPLICASGLOBAL · ONE PER NODEnode 1tasktasknode 2tasknode 3node 4node 1tasknode 2tasknode 3tasknode 4task
Figure 2. Replicated services run N copies wherever they fit; global services run exactly one copy on every node.
Swipe sideways to see all of it, or tap to open it full screen.

Placement

docker node update --label-add zone=ap-south-1a --label-add disk=ssd node-2
docker service create --name db \
  --constraint 'node.labels.disk==ssd' \
  --constraint 'node.role==worker' \
  --placement-pref 'spread=node.labels.zone' \
  --reserve-memory 1g --reserve-cpu 1 \
  --limit-memory 2g --limit-cpu 2 \
  postgres:16
  • Constraints are hard requirements: node.role, node.hostname, node.id, node.platform.os/arch, node.labels.* (set by managers — trustworthy) and engine.labels.* (set on the daemon).
  • Placement preferences (spread=) are soft: spread tasks evenly across a label's values — e.g. across AZs for resilience.
  • Reservations affect scheduling: a node must have that much unreserved CPU/memory. Limits cap usage at runtime.
  • --max-replicas-per-node limits how many tasks of a service land on one node.

A task stays Pending when no node satisfies its constraints or has enough unreserved resources — docker service ps --no-trunc shows "no suitable node".

Enlarge diagramnode-1zone a · ssdnode-2zone b · ssdnode-3zone b · hddnode-4zone c · ssdconstraint node.labels.disk==ssd · preference spread=node.labels.zoneEligible: nodes 1, 2 and 4. Tasks spread across zones a, b and c.
Figure 3. Constraints are hard rules (only SSD nodes); preferences are soft (spread across zones). A task stays Pending if no node satisfies them.
Swipe sideways to see all of it, or tap to open it full screen.

Node availability

Availability Effect
active Accepts new tasks
pause Keeps existing tasks, accepts no new ones
drain Shuts down its tasks; the orchestrator reschedules them elsewhere; accepts no new tasks
docker node update --availability drain node-3    # before maintenance/patching
docker node update --availability active node-3

Draining managers (in a cluster with enough workers) keeps application load and noisy neighbours away from the Raft quorum — a common production practice.

Enlarge diagramdrain node-3docker node update--availability draintasks moverescheduled on othernodespatch / rebootnode-3 is emptyset activeit takes new tasksagain1234Swarm does not move existing tasks back automatically; docker service update --force rebalances.
Figure 4. Draining a node before maintenance: its tasks are stopped and rescheduled elsewhere, and it accepts no new work.
Swipe sideways to see all of it, or tap to open it full screen.

Note: when a drained node returns, Swarm doesn't automatically rebalance existing tasks onto it; new or updated tasks will use it. Force a rebalance with docker service update --force web.