Engineering StrategyAug 14, 20264 min read

Error Budgets Only Work If Something Stops

By Maplecode

Error Budgets Only Work If Something Stops

Most organisations that adopt service level objectives end up with a dashboard nobody acts on. The targets exist, the graphs are green or amber, and delivery continues at the same pace regardless.

The mechanism was never the measurement. It is the agreement about what happens when the budget is gone, made before anyone is under pressure.

The budget is a decision rule

An availability target of 99.9% over thirty days permits roughly forty-three minutes of failure. That remainder is the budget, and its purpose is to answer one question: are we allowed to take risk this week?

With budget left, ship. Deploy on Friday if that suits you, run the migration, take the calculated chance. Budget spent, and feature work stops until reliability is restored. Not "we should be careful" — stops, with the team's next work item being the cause of the burn.

That consequence is the entire mechanism. Without it you have measurement without feedback, which changes nothing.

Choose objectives users would notice

The common mistake is measuring what is easy to instrument. Server CPU, pod restarts, average response time. None of these tell you whether anyone had a bad experience.

Pick the journey instead. Can a customer complete checkout. Does a search return in under a second at the 95th percentile. Do overnight batches finish before the business day. These are harder to instrument and they are the only ones worth stopping work over.

Averages hide the thing you care about. An average response time of 300ms is consistent with a tenth of users waiting four seconds, and those users are the ones who leave.

Set the target from the cost of failure

Reliability targets are usually chosen by adding a nine to whatever sounded acceptable. Each nine multiplies cost — in redundancy, in review process, in delivery speed — and it should be justified per service.

An internal reporting tool can be down for an hour on a Sunday and nobody is harmed. A payment path cannot. Setting both to the same target means overspending on one and underprotecting the other, and it guarantees the budget is ignored, because a team held to a target that does not match their service's real importance learns to disregard it.

The useful conversation is what an hour of downtime actually costs, in revenue, in support load, in obligation. That number picks the target.

The failure mode: the budget nobody can spend

Two symmetrical failures.

A target set so high that normal operation consumes the budget means the team is permanently in freeze, so the freeze stops meaning anything and gets waived. A target so loose that it is never threatened provides no signal at all, and its green dashboard is used as evidence that reliability is fine while users complain.

A healthy budget is partially consumed most months. If yours is always full or always empty, the number is wrong, and adjusting it is maintenance rather than failure.

Make the consequence real before you need it

The freeze has to be agreed by whoever owns the roadmap, in advance, in writing. Discovering that a product director does not accept it during an incident month is the normal outcome of skipping this.

Write down what a freeze includes and excludes — reliability work continues, security patches continue, committed customer deadlines are handled how — and who can override it and on what basis. An override that requires a named person to sign is a functioning process. An override anyone can grant informally is the absence of one.

When we would advise against this

If the service has no meaningful traffic, an SLO is theatre. You cannot measure a percentile on twelve requests a day, and the effort is better spent on monitoring that tells you it is down at all.

If your organisation will not accept the freeze, do not implement error budgets. Implement alerting on user-visible failure instead, and be honest that reliability is a priority call rather than a governed one. That is a legitimate position; pretending otherwise produces a compliance ritual and a team that has learned the numbers do not matter.

Alert on burn rate, not on breach

An alert that fires when the budget is exhausted arrives too late to act on. What you want is notice that the current rate of failure will exhaust it.

The practical shape is two windows. A short one — an hour or so — catching fast burns that indicate an active incident, wired to a page. And a longer one, a day or several, catching slow degradation that nobody would otherwise notice, wired to a ticket rather than a phone.

This distinction matters because the two failures need different responses. A fast burn is an incident. A slow burn is a design problem, and paging someone at 3am about a leak that has been running for a week teaches them to ignore the alert.

Where we start

One service, one user journey, one objective with a target derived from what failure costs. A dashboard showing budget remaining rather than uptime achieved. A written consequence signed by the person who owns delivery. Then review the target quarterly and expect to move it — the first one is a hypothesis.

One organisational note. Error budgets only function where the same group owns both reliability and delivery. Split those across two teams and the budget becomes a negotiation between departments rather than a decision rule, and the team that owns delivery has no incentive to respect a constraint the other team measures.

Related: delivery automation and managed services.

Start here

Let's build what's next.

Tell us where you are and where you want to be. We'll bring the engineering, the AI, and the governance to get you there.