An error budget is a number, not a feeling

Most AI workflow teams already sense when something is off: more escalations than usual, a client asking why a draft was wrong, a reviewer pushing back twice in a row. The problem with sensing it is that nobody agreed beforehand what 'too many' means, so the response is inconsistent. One week a spike gets a shrug, the next week an identical spike triggers a scramble.

Site reliability engineering solved a version of this problem for software uptime by picking a service level objective, such as 99.9% successful requests over a quarter, and treating the remaining 0.1% as a budget the team is allowed to spend. The same structure works for an AI workflow: define one outcome that matters (a correctly routed ticket, a decision packet accepted without revision, a drafted response sent without edits), set a target rate for it, and treat the gap as spendable, not as an emergency every time it moves.

Pick the metric before picking the number

The number only means something if the metric underneath it is real. Pick the outcome a business owner already cares about, not an internal model metric nobody outside engineering recognizes. For a document-review workflow that might be 'percentage of drafts approved without a human rewrite.' For a lead-qualification workflow it might be 'percentage of routed leads confirmed as correctly scored by sales.'

Once the metric is fixed, set the target from the actual cost of a miss, not from what sounds impressive. A workflow that drafts internal notes can tolerate a much lower success rate than one that writes back to a customer record or triggers a payment. Write the target down with the owner's name next to it, because a target nobody signed off on gets renegotiated the first time it's inconvenient.

  • Use an outcome a non-technical stakeholder already tracks, not a model-internal score.
  • Set the target from the cost of a miss for that specific workflow, not a generic benchmark.
  • Name the person who owns the target so it can't be quietly redefined later.

Spend the budget on launches, not on routine operation

The budget's purpose is to make a concrete trade visible: every risky change (a new prompt version, a new data source, a widened permission, a new automation turned on without review) draws down the same pool that routine operation already uses. If the workflow is running near its target with little room left, that is the signal to slow down releases, add a review step back in, or roll a recent change back, before the next change is allowed to ship.

This only works if the rule is enforced the same way every time. If the budget runs out and the team ships anyway because a deadline is close, the budget stops meaning anything and reverts to the same case-by-case argument it was meant to replace. Google's own description of this mechanism is explicit that it depends on the team with the stop authority actually being willing to use it.

Review the budget on a fixed schedule, not only after a bad week

Check the budget on a calendar cadence, weekly or monthly depending on workflow volume, rather than only when something goes visibly wrong. A workflow that is quietly drifting toward its limit needs the same attention as one that just had a bad day; reviewing only after incidents means the team always finds out late.

Over a few review cycles, the budget also becomes a planning input. A workflow that keeps most of its budget unspent can absorb a larger change, like a new model version or an added data source. One that is consistently near its limit needs its current version stabilized before anything else gets added, which is a more useful planning conversation than debating any single failure in isolation.