Coupling shows up gradually, not on day one
Most workflows start with a single call to a single model provider, and that is a reasonable way to ship a first version. The coupling accumulates afterward: a prompt tuned around one model's formatting quirks, a tool-calling schema written to one provider's function-call syntax, retry logic that assumes one provider's specific error codes, and evaluation scripts wired to one SDK's response shape.
None of these decisions look like lock-in individually. Together they mean that swapping providers, or even swapping model versions from the same provider, requires touching code across the workflow instead of changing one configuration value.
Put the interface below the workflow logic, not inside it
The fix is architectural, not a policy about which vendor to prefer. Application and workflow code should call an internal interface for completions, tool calls, and structured output, and that interface should be the only place a provider SDK is imported.
AWS's guidance for scaling agentic AI across an enterprise makes this same point at the platform level: standardize the shared control planes, such as identity, policy enforcement, observability, and routing, below the application layer, and leave room for teams to change how agents are built and executed above it. The same boundary applies inside a single workflow, just at smaller scale.
Open-source gateways such as LiteLLM exist specifically to sit at this boundary. They expose one interface, typically shaped like the OpenAI chat completions API, and translate it to whatever provider is configured behind it. Whether a team adopts a tool like that or writes a thinner internal wrapper, the requirement is the same: one seam, not scattered provider calls.
- One internal function or service for completions, not direct SDK calls from workflow code
- Provider, model name, and credentials read from configuration, not hardcoded
- Tool-call and structured-output schemas defined once, translated to each provider's format at the seam
Prompts and evaluation sets are workflow assets, not provider code
A prompt that only works because of one model's specific formatting habits is a hidden dependency. Write prompts to a clear, provider-neutral specification of the task and the required output, then test that specification against each provider you might use, rather than hand-tuning wording to one model's preferences.
The same applies to evaluation. A golden set of representative cases with expected outcomes should be able to run against a new model candidate and produce a pass or fail signal before that model is trusted with production traffic. If the evaluation harness itself only speaks one provider's response format, it becomes another thing to rebuild every time the model changes.
Decide the fallback path before an outage forces it
Providers have outages, deprecate model versions, and change rate limits without much notice. A workflow with a clean interface boundary can define what happens next: fail closed and queue the work, fall back to a secondary provider for that task, or route to a smaller model with a narrower capability for degraded but continued operation.
Deciding this during an incident is expensive and rushed. Deciding it as part of the interface design, alongside the normal request path, turns a provider disruption into a routing decision instead of a production emergency.
Portability is not free, so scope it to what the workflow needs
None of this means chasing every new model release or running a live bake-off on every workflow. The interface boundary is worth building because it is cheap when it is designed in from the start and expensive to retrofit once dozens of call sites assume one provider's behavior.
The practical scope is narrower than full model-agnosticism everywhere: identify which workflows are consequential enough that a provider disruption would be costly, put the interface boundary there first, and extend it to lower-stakes workflows as time allows.