Opscraft
SRE & incident management · Lesson 2 of 7 · 3 min read

Error budgets and policies

Using the budget

The error budget turns reliability into a shared decision tool between product and engineering:

  • Budget healthy → ship features, run experiments, do risky migrations.
  • Budget burning fast or exhausted → error budget policy kicks in: freeze non-critical releases, prioritise reliability work, require extra review for changes, until the budget recovers.

A policy should be agreed in advance with product leadership, documented, and applied consistently — otherwise it's ignored in the first crisis.

Enlarge diagramBudget healthyBudget burning or goneShip features fastRun experimentsDo risky migrationsFreeze non-critical releasesReliability work firstExtra review for changes
Figure 1. The error budget as a shared decision tool between product and engineering.
Swipe sideways to see all of it, or tap to open it full screen.

Burn rate

Burn rate = how fast you're consuming the budget relative to plan. Burn rate 1 = exactly exhausting the budget at the end of the window. With a 30-day window, a burn rate of 14.4 sustained for 1 hour consumes 2% of the monthly budget.

Enlarge diagram100%030 daysburn rate 1: lasts exactly 30 daysburn rate 14.4: gone in ~2 days → pagebudget remaining over time
Figure 2. Burn rate: how fast you are using the month's error budget. At 14.4× the whole month's budget would be gone in about two days.
Swipe sideways to see all of it, or tap to open it full screen.

Multi-window, multi-burn-rate alerts (Google SRE workbook) balance fast detection and low noise:

Severity Burn rate Long window Short window Budget consumed
Page 14.4 1 h 5 m 2%
Page 6 6 h 30 m 5%
Ticket 1 3 d 6 h 10%

The short window ensures the alert resets quickly when the problem stops. PromQL sketch for a 99.9% SLO (budget 0.001):

Enlarge diagram1 h windowburn > 14.4×5 min windowburn > 14.4×ANDpage on-call
Figure 3. Multi-window alerts: page only if both a long and a short window are burning fast. The long window proves it is real; the short one lets the alert clear quickly.
Swipe sideways to see all of it, or tap to open it full screen.
(
  sum(rate(http_requests_total{service="checkout",status=~"5.."}[1h]))
    / sum(rate(http_requests_total{service="checkout"}[1h]))
) > (14.4 * 0.001)
and
(
  sum(rate(http_requests_total{service="checkout",status=~"5.."}[5m]))
    / sum(rate(http_requests_total{service="checkout"}[5m]))
) > (14.4 * 0.001)

Tools like Sloth, Pyrra or OpenSLO generate these rules from an SLO definition; Grafana, Datadog and CloudWatch Application Signals provide SLO tracking.