The model is the cheapest part to operate
Inference is the cost most teams can quote from memory, because it arrives on a bill with a number on it. It is also the line item most likely to fall over time as providers compete and smaller models become adequate for the task.
The costs that persist sit around the model: the connectors that fetch evidence, the rules that decide what happens next, the queue where people review consequential actions, and the tests that prove all of it still works. A decade of engineering experience with machine learning systems points the same way — the model code is a small fraction of what has to be built and maintained.
Evidence plumbing does not hold still
An AI workflow depends on reaching policies, tickets, records and documents at the moment a decision is made. Every one of those sources is owned by somebody else, and none of them will consult you before changing.
Budget for connector maintenance as a running cost rather than a one-off integration task. The workflow does not fail loudly when a source drifts; it usually starts producing confident output from incomplete evidence, which is harder to notice and more expensive to unwind.
- Upstream schema and API version changes
- Credential rotation and shifting permissions
- Document formats and naming conventions
- New systems added to the process after launch
Review time is a staffing line, not a safety note
If a person approves consequential actions, that review is a recurring labour cost that scales with volume. A pilot handling twenty cases a week hides this completely; the same design at two thousand cases a week is a staffing decision.
Measure time to decision and the proportion of cases that need review, and treat a falling review rate as the goal rather than a nice surprise. If the rate never falls, the workflow has moved effort rather than reduced it, and the business case should say so plainly.
Evaluation is a standing commitment
The test cases assembled before launch represent the situations understood at the time. Production supplies new ones continuously, and each genuine edge case is worth adding to the set that gets re-run.
Every later change — a new model version, a revised prompt, an adjusted business rule, a reconnected integration — needs that set run again. Without it, teams end up changing things they cannot verify, and confidence in the workflow decays faster than its accuracy does.
Something unowned quietly rots
Name the person accountable for the workflow after launch, and write down what happens when an integration fails outside working hours, who decides whether to pause it, and how a case gets completed by hand while it is down.
Set a retirement condition too. A workflow built for a process that no longer exists is a liability that still holds credentials and still writes into live systems. Deciding in advance what would make it worth switching off is cheaper than discovering it later.