One Harness, Many Models: Providers, Auth, and Compatibility
Pi's provider layer answers a narrower question than the agent loop: given a model choice, how should a request be authenticated, represented, streamed, priced, and continued? This chapter examines @earendil-works/pi-ai and the coding-agent model runtime so you can choose a provider without mistaking a shared interface for identical behavior. The goal is practical: understand which decisions belong to a provider, which belong to a model record, and which still belong to the host running Pi.
In the previous turn model, the agent reached a provider boundary after projecting its richer transcript into messages the model could understand. That boundary is where portability becomes real engineering. Anthropic Messages, OpenAI Responses, OpenAI-compatible local servers, and other APIs can all carry an agent request, but they differ in authentication, tool schemas, thinking controls, cache behavior, and response details. Pi's design does not erase those differences. It gives them an explicit place to live.
A provider owns the route, not just the URL
The central object in Pi's AI layer is a Provider. A provider owns an identity, a model catalog, authentication methods, and the stream behavior used to reach those models. A Models collection holds providers and routes a request by the model's provider and identifier. The collection supplies convenient operations such as model lookup, availability checks, streaming, completion, login, logout, and refresh; the provider remains the runtime unit that knows how the request actually works.
That division is important because a URL is not a provider contract. Two endpoints may accept a broadly similar request shape while differing in how they name the maximum output field, whether they accept a developer role, how they encode reasoning, or whether they support strict tool schemas. The provider owns the API implementation. The model record describes the capabilities and limits that the agent should use when preparing a request. The collection connects the two and applies authentication before dispatch.
Pi's createProvider() makes this ownership visible. A provider can use one API implementation for all of its models, or it can hold a map keyed by model.api for a mixed-API provider. The latter matters for gateways and providers that expose different model families through different wire protocols. The request still enters through one Models collection, but dispatch can select the correct implementation from the model metadata rather than from a global registry hidden elsewhere.
Dynamic catalogs follow the same rule. A provider may begin with a static baseline and overlay models fetched from a live source. Pi can restore a provider's previously stored catalog before attempting network access, then let the provider publish a refreshed catalog. Publication is generation-checked: if a newer refresh or provider replacement supersedes an older operation, the stale operation cannot publish its result afterward. A failed refresh can therefore leave the last known model list in place instead of turning a temporary catalog problem into a loss of all model knowledge.
This is a better abstraction than a universal list of model names. The model list is useful only when it carries the facts needed by the loop. In Pi, a model record includes its provider and API, whether it supports reasoning, what it can accept as input, its context window and maximum output, cost rates, default sampling parameters, optional headers, and compatibility settings. Those fields are operational inputs, not descriptive decoration.
Notice what this ownership model prevents. A model identifier by itself is not a complete route: two providers can expose similarly named models while using different credentials, endpoints, and wire protocols. Pi's lookup includes the provider and model together, so a request cannot accidentally drift to whichever global adapter happens to recognize the name. A synchronous lookup reads the last-known catalog; an availability query adds the separate question of whether the provider is currently configured for use. This lets tooling inspect model metadata without pretending that an unconfigured model is ready to receive a request.
The same distinction matters for dynamic catalogs. A live provider can restore its last stored list first, then refresh it when network access and credentials permit. If a refresh is interrupted or fails, the provider can retain its previous list. If a newer refresh supersedes an older one, generation checks prevent the older result from publishing after it has become stale. The catalog is therefore treated as provider-owned state with a lifecycle, not as a one-time download that the rest of the application must trust forever.
There is also a useful separation between an API implementation and a provider. Several providers can share an API implementation because they speak the same wire protocol, while one provider can dispatch different model families through different implementations. That arrangement avoids two opposite errors: duplicating identical serialization logic for every provider, or forcing a mixed provider to pretend that all of its models have one request shape. The common Models surface remains stable while the dispatch decision stays close to the data that justifies it.
For example, reasoning support changes what the agent can ask the provider to do. A model may expose ordinary thinking levels while another exposes additional levels through an explicit map. A model that accepts images can receive image content; a text-only model cannot be treated as equivalent. A context window and output cap affect compaction and token budgeting. Cost metadata determines how usage is accounted for. If the model record says only “Claude” or “local model,” it has not given the loop enough information to behave safely or predictably.
Pi also separates the common catalog API from the API-specific option types. A caller can use a provider-neutral streamSimple operation for common controls such as reasoning. When a caller needs a protocol-specific option, the model's API can be narrowed and the full adapter-specific type used. This preserves a common path for the ordinary case without hiding the escape hatch required by a real provider integration.
The same principle explains lazy provider implementations and selective imports. The AI package can expose a core collection without forcing every provider catalog and SDK into every application. Provider factories can load a catalog and a lazy API wrapper; a request loads the concrete implementation when needed. That is a packaging consequence of ownership: an application that uses one provider should not have to accept every provider's runtime surface merely because the common abstraction exists.
Authentication and compatibility are part of model choice
Once a model is selected, Pi must resolve credentials before dispatch. The resolution path has more structure than “read an environment variable.” A caller may supply an explicit key or environment overlay for one request. Otherwise the provider's credential store is consulted. If there is no stored credential, a provider-owned API-key resolver can inspect ambient configuration such as environment variables, AWS profiles, or cloud application credentials. OAuth-capable providers use their own login, refresh, and conversion methods.
The ownership rule is deliberate: a stored credential owns its provider. If a stored OAuth credential needs refreshing, Pi refreshes it through the provider's OAuth handler and persists the rotated result. It does not silently fall back to an environment key after a refresh failure. Silent fallback would make a request appear to succeed under a different identity, endpoint, quota, or policy than the user selected. A failed refresh is an authentication failure that needs re-login or an explicit change of configuration.
Credential mutation is serialized per provider. The store contract exposes a read, a list of non-secret credential metadata, a read-modify-write operation, and deletion. The read-modify-write path is the only write path, and concurrent operations for one provider wait on a promise chain. OAuth resolution adds a second check under that lock: a request that notices an expiring token re-reads the current credential before refreshing. If another request has already rotated it, the second request uses the new value rather than refreshing the old token again.
The coding-agent runtime adds its own non-persistent runtime override for cases such as a temporary key supplied by an embedding application. The broader order remains a policy question at the boundary: explicit request values should win when deliberately supplied, then stored credentials, then ambient configuration or a custom-provider fallback. A Pi auth file is created with user-only read/write permissions according to the provider documentation, but that file still contains sensitive material and belongs inside the host's security policy.
Provider compatibility enters after authentication has been assembled. Pi merges provider auth, model-level headers, and explicit request headers, then allows a final request transform to add or remove headers before the provider dispatch. The model can override the provider's base URL or compatibility settings. This enables a proxy route or a local endpoint without forcing the provider abstraction to pretend that the proxy is identical to the upstream service.
Consider an Ollama, vLLM, or LM Studio endpoint that speaks an OpenAI-compatible protocol. It may accept the same broad chat format while rejecting the developer role used by reasoning-capable models, using max_tokens instead of max_completion_tokens, or ignoring a provider's reasoning field. Pi lets the configuration mark those differences. A compatibility record can say to send the system prompt as system, suppress unsupported reasoning controls, choose the right token field, omit unsupported streaming usage, or select the appropriate thinking format. The adapter then serializes the request according to those facts.
This is why the compatibility field is not a collection of arbitrary toggles. Each setting answers a concrete boundary question: what does this endpoint accept, and what should Pi do when the common model representation does not map directly to it? The same model-facing request can be transformed differently for OpenAI Responses, OpenAI Chat Completions, Anthropic Messages, or a local server. A custom endpoint that happens to return text is not automatically compatible with tool calling, thinking replay, cache markers, or strict schemas.
The model record also carries cache and usage consequences. A session identifier can be passed to providers that use it for prompt caching, routing, or affinity. Cache retention is a preference that each provider maps to its own supported representation; a requested cache mode is not proof that the upstream will produce a cache hit. Cost calculation uses the model's rates and reported usage, including cache reads and writes where the provider reports them. A context window and maximum output limit are used to leave room for the answer and to decide when compaction is necessary.
Custom model configuration is intentionally flexible but not consequence-free. A models.json entry can point at a local or proxy endpoint, define a model identifier, describe reasoning and input capabilities, set limits and costs, and add sampling parameters that Pi's common type does not name. Provider and model overrides can route built-in models through a proxy or adjust a built-in model's metadata without replacing the whole catalog. Values can come from literals, environment interpolation, or shell commands. Shell-command values are resolved at request time, and Pi does not invent a universal cache or stale-value policy for arbitrary commands; the wrapper supplied by the host must decide how to handle slow or transient secret sources.
Changing models during a conversation therefore has two meanings. At the control level, Pi can record a model change and send later requests through the new provider. At the message level, the conversation must be translated for the new API. User and tool-result messages can pass through. Assistant messages from the same provider/API can be preserved more directly. Assistant messages from another provider may need their thinking blocks converted into tagged text, while ordinary text and tool calls remain available. This preserves useful continuity, but it is not semantically identical to replaying the original provider's native transcript.
The same limit appears when reasoning is involved. A provider may attach opaque thinking signatures or native effort metadata needed for replay. Another provider may not understand those fields, so Pi converts the thinking content into a form that the next provider can consume as text. That keeps the conversation transferable, but it changes the representation and may change what the next model can use as native reasoning context. A model switch is therefore a controlled handoff, not a claim that every provider sees precisely the same internal history.
Pi's provider layer succeeds when it standardizes the mechanics that should be common—lookup, routing, authentication application, streaming events, normalized usage, and model selection—while leaving protocol-specific truth at the adapter and model boundaries. The agent loop can ask for a tool-capable response without knowing how each provider encodes it. The host can add a local endpoint without forking the loop. But neither benefit removes the need to describe capabilities honestly. Once the model route is chosen, the next question is who supplies the tools, instructions, and workflow policy that the model is allowed to use.