A workflow is not one task

A typical business workflow strings together several distinct jobs: pull a field out of a document, classify a request, draft a response, decide whether a case needs escalation. Each of those steps has a different error cost and a different amount of genuine ambiguity.

Teams often pick one model for the whole pipeline because it is simpler to build and reason about. That simplicity has a price. A model capable enough for the hardest step in the chain is usually far more than the easy steps need, and every call to it costs more and takes longer than it has to.

Match model strength to step difficulty

Anthropic's own guidance on choosing a model frames this as a starting-point decision: begin efficiency-first with a smaller, faster model for high-volume or straightforward tasks, and reserve the strongest model for complex reasoning, nuanced judgment, or work where accuracy outweighs cost. The same guidance describes combining models directly, pairing a lower-cost model with a frontier model so an orchestrator delegates bulk work to cheap workers and escalates only the hard decisions.

That framing maps cleanly onto a business workflow. A step that extracts a date, a dollar amount, or a category from a structured document is a volume task with a narrow answer space. A step that drafts a client-facing explanation, resolves a policy ambiguity, or recommends an exception usually carries more judgment and a higher cost if it is wrong. The first kind of step is a candidate for a smaller model; the second usually is not.

  • Extraction and field-level classification: candidate for a smaller, faster model.
  • Routine formatting, summarization, or routing: candidate for a smaller model with spot checks.
  • Drafting consequential communication or judging an ambiguous case: keep on the stronger model.
  • Escalation and exception decisions: keep on the stronger model, with human review already in the loop.

Cascades make the savings structural, not accidental

This is not a new idea in the research literature. The FrugalGPT paper studies exactly this pattern under the name LLM cascade: route each query through smaller, cheaper models first and escalate to a larger model only when the cheaper response looks unreliable. The paper reports cost reductions of up to 98% while matching the accuracy of always calling the strongest model, and in some configurations beating it at the same cost.

The business version of a cascade does not need research-grade routing logic to be useful. A fixed rule, such as always extracting with a small model and only promoting a case to the larger model when a confidence score, a validation failure, or a specific category triggers it, captures most of the benefit without new infrastructure.

Evaluate the swap, do not assume it

Downgrading a step's model is a change like any other in a production workflow, and it needs the same evidence before and after. Run the same evaluation set the step was already measured against, and compare accuracy, not just cost, before the smaller model takes over that step permanently.

Keep the comparison narrow to the step being changed. A model swap that looks neutral on an aggregate workflow metric can still hide a real accuracy drop on the one step it touched, especially if that step feeds a downstream step that was tuned around the previous model's output style.

Revisit the split as models change

Model generations move fast enough that a split that made sense six months ago can be stale today. A smaller model might now handle a step that previously needed the strongest one, or a new release might shift the cost-performance curve enough to justify moving a step the other way.

Treat the per-step model assignment as a line item to review on the same cadence as the rest of the workflow's production metrics, not as a decision made once at launch and left alone.