The Model Gateway That Treats Traffic as Evidence
Experiential is an open-source gateway and router for agent workflows. The question in this opening chapter is why the project joins familiar model APIs to trace ingestion, evaluation, routing, and accounting instead of treating the gateway as a thin proxy. That question matters because an agent's model choice is rarely a one-time decision: the request may need tools, a particular context window, a continuity-preserving model, a cost ceiling, or a provider that can honor a capability the caller assumes. The reader's task is to understand the project as a trace-to-serving system and to see what its design makes possible—and what its evidence still cannot prove.
The problem is not choosing a model once
A conventional model integration often has a simple shape: an application sends a request to a named model and receives a response. That model is useful as a starting point, but it hides the decisions that become important when the application behaves like an agent. An agent may call a model repeatedly, carry forward tool results, switch between tasks with different stakes, and accumulate usage that matters to a team rather than to one isolated request. The model name is only one part of the operating decision.
Consider a support agent that must answer a customer, call a retrieval tool, and continue after the tool returns. The right choice is not necessarily the cheapest model or the model that won the last benchmark. It may be the candidate that accepts the tool protocol, can fit the requested output budget, preserves the relevant conversation semantics, and remains available inside the identity's allocation. If a fallback model cannot honor one of those conditions, a superficially successful API call may still be the wrong result. If a provider fails after it has emitted meaningful output, switching models can create a second answer rather than a recovery. These are operational questions about authority and state, not merely questions about model intelligence.
Experiential addresses this broader problem through three connected capabilities. It offers one OpenAI-compatible API for hosted, bring-your-own-key, and local models. It gives operators a way to control which identities and agents can use which models and how much they may spend. It also turns production traffic into material for a custom router or model optimization workflow. Those capabilities are not three unrelated product surfaces. They are three points on one control loop: serving creates behavior, behavior becomes evidence, evidence informs a policy, and the policy changes future serving.
That framing is more precise than calling Experiential an “AI gateway.” A gateway answers the immediate question, “May this request reach a provider, and under what contract?” A router answers, “Which eligible model should handle it?” An optimization workflow asks, “What evidence justifies the relationship between request characteristics and that choice?” Experiential keeps those questions connected while preserving their different responsibilities. The online gateway cannot simply trust a learned choice without checking identity, capability, budget, and model metadata. The offline optimizer cannot treat every generated or judged example as if it were observed traffic.
This is the practical tension behind the project. The more an organization wants model choice to adapt to real work, the more it needs a reliable account of what “real work” means, how candidate responses were compared, and whether the resulting policy still describes the models actually available at serving time. A fast selector without evidence can be opaque. A carefully evaluated policy without serving authority can be bypassed, mispriced, or applied to a route that cannot honor the request. Experiential's architecture is an attempt to make both sides answerable to the same evidence and identity boundaries.
It also changes what “improvement” means. A project can improve an offline score by changing a candidate, a judge, a task mix, a retrieval set, or a cost assumption. It can improve an online response rate by relaxing a capability check or by making a fallback more permissive. Neither change is automatically an improvement for the people using the agent. By binding evidence, policy, and serving identity, Experiential creates places where those choices can be inspected separately. The architecture does not remove the need for judgment; it makes it harder to hide which judgment was made.
Experiential's closed loop
The loop begins with production or imported traffic, but not with an assumption that any export is immediately usable. A declared trace source is normalized into a canonical representation with source identity, model evidence, and explicit normalization issues. The resulting records are preserved as immutable artifacts. From them, the project can derive representative tasks and the retrieval material used to ground later simulation. This first stage answers a foundational question: what exact behavior is the rest of the system allowed to learn from?
That question has a practical consequence during a rebuild. If the source, normalized bytes, issue list, model evidence, or producing revision changes, the resulting artifact identity should change or the existing artifact should be verified as an exact replay. A later policy can then point back to the particular evidence graph it used instead of relying on a mutable “latest data” label. This does not guarantee that the source is complete or representative. It does make incompleteness visible as a property of the evidence rather than an invisible change in the input.
The second stage uses that evidence to extend the workload without rewriting its origin. Experiential's retrieval contract is based on real action-to-observation transitions: a visible assistant action or tool call must be followed by a genuinely observed user message or environment result. A terminal assistant response with no subsequent observation is not treated as a complete transition. The read-only index can support a grounded world model, but generated world-model predictions, simulator rollouts, judgments, evaluations, and manual examples are not allowed to become new grounding data. That separation is the difference between using a simulation to ask a counterfactual question and allowing the simulation to manufacture its own history.
The third stage turns evidence into a router policy. Candidate models are evaluated against a combined plan containing fit and held-out cells. Fit evidence is completed first; the router bank and policy are then locked. Only after that lock does the workflow open held-out evidence and produce its report. The order matters. If the policy could change after seeing held-out outcomes, the report would no longer be a clean test of a previously chosen policy. World-model fidelity is also kept separate from router fitting, because confidence in a simulator and authorization of a routing policy are different judgments.
The fourth stage is online serving. A frozen policy is loaded with its bank, feature definition, pricing snapshot, model identities, and embedder identity. At request time, the router can select a model using the policy and preserve a bounded sticky decision for an episode. The gateway then applies the authority that the offline artifact does not own: authentication, grants, capability checks, cost reservation, provider attempts, failover rules, settlement, and content-free usage accounting. A compatible API makes the caller's integration easier, but it does not make all providers semantically identical. Unsupported controls may need to be narrowed, translated, or dropped with disclosure; unknown capabilities and prices are not supposed to be inferred from a provider's name.
The boundary is visible even before a model request begins. Gateway startup and readiness do not make a provider call; provider work begins only after an authorized request crosses the serving boundary. This arrangement lets configuration and authority be checked without quietly spending provider budget. It also means that a “working gateway” and a “ready provider route” are different conditions. The project requires explicit identities, grants, alias revisions, catalog snapshots, and credential references before a route is considered usable. The distinction is easy to miss in a simple proxy, but important when the same process is expected to enforce budgets and preserve an audit trail.
The loop can be represented simply:
The return path is not merely a slogan. Completed routed interactions can be sealed into an immutable runtime trace prefix and prepared for a later model-optimization workflow. That workflow still has its own gates: it needs completed interactions, a bounded training configuration, a conservative full-schedule cost estimate, and recursive verification before a result can be registered. The existence of a return path means the system can preserve a learning loop; it does not mean that every response is automatically good training data or that training has already improved a model.
What the loop can—and cannot—prove
The architecture makes several kinds of judgment more explicit. It can preserve which source and code revision produced a dataset, which real transitions grounded retrieval, which evidence entered a fit partition, which policy was locked, and which model identity was authorized at runtime. It can separate a physical provider attempt from a logical request and retain cost information when retries or uncertain failures make a simple success-only ledger misleading. It can also keep prompts, responses, raw tool arguments, virtual keys, and provider secrets out of the gateway's durable content-free accounting state.
Those mechanisms do not eliminate uncertainty. A trace corpus can be faithfully normalized and still be narrow, biased toward the tasks an organization happened to record, or incomplete about the outcomes that matter. Similarity retrieval can find a useful transition without proving that it represents every future case. A grounded rollout can make candidate comparisons possible without becoming a production observation. A held-out report can show what happened under one plan, candidate set, judge, budget, and evidence split without establishing that the same policy wins across every workload.
Three statements that sound similar therefore have different evidentiary force. “This transition was observed” is a statement about provenance. “This candidate produced a response in a grounded simulation” is a statement about a bounded counterfactual run. “This policy served a completed request” is a statement about an online interaction whose identity, route, and economics can be recorded. Experiential's value is partly the refusal to collapse these statements into one generic category called data. Each can support a different decision, and each needs a different qualification.
The repository is unusually explicit about this distinction in its release scope. Its router and sandbox evidence is deterministic and uses injected clients; its live provider matrix remains credential-gated where no live run occurred. The native gateway benchmark reports measurements for a stated machine and test setup, not a universal service-level objective. The release also makes no claim of real managed Tinker training or a trained-versus-base quality improvement. These are not footnotes that weaken the project. They define the boundary within which its mechanisms can be interpreted honestly.
The central idea, then, is not that Experiential removes the judgment from model selection. It is that the project gives the judgment a sequence, an owner, and a durable boundary. Traffic is not automatically truth; it is first made identifiable. Simulation is not automatically evidence of reality; it is grounded and kept separate. A learned policy is not automatically authority; it is activated only alongside explicit model and catalog identity. A successful API call is not automatically a validated improvement; it remains part of an accounting and evidence lifecycle.
That is why the next question is unavoidable: if later decisions depend on traffic, what makes a trace trustworthy enough to enter the loop? The answer begins with the evidence lifecycle itself—how Experiential declares a source, records what normalization rejected, and binds the resulting records into a project graph that later stages can verify.