Cost per call is the wrong denominator

Most teams start by watching the API bill: tokens in, tokens out, dollars per thousand calls. That number is easy to pull from a provider dashboard, but it does not say anything about whether the workflow is worth running. A step that costs less per call but needs three retries to produce a usable result is not cheaper; it is just cheaper to measure wrong.

The AWS Cloud Financial Management team makes the same point in its generative AI cost guidance: total cost of ownership should be tied to business KPIs, not just infrastructure spend, and it specifically recommends metrics like cost per completed text summary rather than raw token cost.

Build the cost-per-outcome number

Pick the unit of work the business actually cares about: a resolved ticket, an approved claim, a drafted contract clause, a qualified lead. Then add up everything that workflow run actually spent to reach that outcome, not just the model call: retrieval, embeddings, tool calls, retries, and any human review time attached to the case.

Divide total spend for a period by the number of outcomes the workflow completed in that period, not the number of attempts. A workflow that fails on the first pass and succeeds on a retry still produced one outcome at a cost that includes both calls. If the metric only counts successful first attempts, it will look better than the business actually experiences it.

  • Model and embedding calls for the run
  • Retrieval, vector search, and tool-call costs
  • Human review time attached to the case
  • Retries and failed attempts that preceded the eventual outcome

Separate the cost drivers before you divide

A single blended number is easy to report and hard to act on. The FinOps Foundation's guidance on AI value frames this directly: cost data usually lives in cloud billing, while the outcome it should map to lives in product analytics or a CRM, and the gap between the two is exactly where teams lose track of what they are paying for.

Before dividing, break spend out by driver: model inference, retrieval and vector search, orchestration and tool calls, storage, and human review. That breakdown shows which lever actually moves the cost-per-outcome number when volume grows, instead of guessing from one combined bill.

Put it next to quality, not instead of it

Cost per outcome only means something next to a quality measure. A workflow that gets cheaper by skipping evidence checks or widening its auto-approval threshold is not improving; it is shifting risk somewhere the dashboard does not show yet.

Track the two together on the same cadence you already use for other production metrics: cost per outcome, completion rate, and the human-review or escalation rate, reviewed at the same interval. A workflow is actually getting better only when cost per outcome falls while quality holds or improves. If quality is dropping while cost falls, that is not an improving system, it is a shortcut that has not been caught yet.