HTTP & status codes · Lesson 8 of 9 · 3 min read
Monitoring, SLOs, retries and API design
What to alert on
- 5xx error rate as a percentage of requests — not raw counts (100 errors per minute is a disaster at 200 rpm and noise at 2 million).
- Latency percentiles (p95/p99) — tail latency often degrades before errors appear.
- Saturation — CPU, memory, connection pools, queue depth.
- Per layer: ELB 5xx vs target 5xx; per route for critical endpoints (checkout, login).
- Burn-rate alerts on SLOs (e.g. 99.9% of requests succeed over 30 days) rather than static thresholds.
Retry policies
- Retry only idempotent requests (GET, PUT, DELETE) — or POSTs with idempotency keys.
- Retry on transient failures: 502, 503, 504, 429 (respect
Retry-After), connection resets/timeouts. Don't retry 4xx like 400/401/403/404 — they'll fail again. - Use exponential backoff with jitter, a small retry limit, and an overall deadline; budget retries across layers (retries at client, mesh and app multiply load: 3 × 3 × 3 = 27×).
- Circuit breakers stop hammering a failing dependency and fail fast.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
Status codes in API design (what good looks like)
201+Locationfor creates;202for async work;204for no-content success.400/422for validation with a structured error body (e.g. RFC 9457 problem details:type,title,detail,instance).401vs403used correctly;404for missing;409for conflicts;429withRetry-Afterfor limits.- Never
200with an error body; never500for client mistakes (it pollutes your error rate and pages on-call for client bugs).