Overview
Experiential connects declared trace ingestion, real-transition retrieval, grounded simulation, evidence-ordered router fitting, and governed gateway serving into one auditable loop. Immutable artifacts preserve provenance; fit evidence locks a policy before held-out testing; runtime identity, capability, budget, safety, and accounting controls constrain each call. Completed interactions can seed bounded future optimization, but simulated rollouts are not observations and verified preparation is not proof of training gain. The approach suits consequential, changing model choices, while its measurements remain bounded by stated workloads, snapshots, setups, and release scope.
Learning outcomes
- Experiential treats traffic as evidence only after declared, normalized, immutable provenance is preserved.
- Grounded simulation extends candidate comparison, but generated rollouts never become observed transitions.
- The router is locked from fit evidence before held-out results open; fidelity remains a separate measurement.
- Online serving keeps learned choices subordinate to identity, capability, budget, safety, and accounting controls.
- Runtime snapshots can seed bounded optimization, but the release does not demonstrate a training gain.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
What makes a trace trustworthy enough for Experiential's routing loop?
A trace is trustworthy enough to enter the loop when its source is declared, its bytes and normalization issues are preserved, and dependent artifacts bind to exact inputs. That establishes provenance, not representativeness.
Can Experiential's world model predict production behavior?
Experiential's world model cannot predict production behavior in general. It generates bounded next observations from real transitions under a specified retrieval, prompt, model, and session contract, supporting controlled comparison rather than production replay.
How does Experiential prevent router evaluation from moving the goalposts?
Experiential completes fit evidence, locks the policy, and only then opens held-out evidence. The result tests a fixed policy under declared candidates, tasks, prices, judges, and protocols, not a universal ranking.
What does Experiential's gateway add beyond model selection?
Experiential's gateway authenticates and authorizes requests, checks capabilities, reserves cost per physical attempt, governs failover and semantic commitment, and records content-free usage. A learned choice remains a constrained recommendation.
Does Experiential prove that runtime feedback improves model quality?
Experiential does not prove that runtime feedback improves model quality. Completed interactions can support immutable, bounded SFT preparation, but the release reports no real Tinker training or trained-versus-base improvement.