The Loop Under Pressure: Safety, Speed, and Feedback

This chapter asks what happens when Experiential's evidence-controlled request path meets operational pressure. The focus is the interaction between identity-scoped guardrails, native streaming, durable accounting, and the return of completed interactions to future model optimization. The reader should be able to explain why enforcement occurs where it does, why Rust improves streaming without removing the SQLite ceiling, and how production feedback can become a bounded training input without becoming a claim that training improved a model.

Safety is a placement decision

Guardrails are not a generic filter placed somewhere around the gateway. In Experiential they are optional, default-off, and scoped to an authenticated organization and identity. That scope matters because the same gateway may serve several identities with different policies. A missing policy leaves the ordinary hot path in place: no classifier call, no stream buffering, and no extra native callback is introduced merely because the gateway supports guardrails.

When a policy is present, input and output enforcement occupy different points in the request lifecycle. Input checks run after authentication and any continuation expansion but before route resolution, ledger acceptance, attempt start, and provider work. This placement lets the policy inspect the canonical request before a model is selected or budget is reserved. If the input is blocked, the request never becomes an admitted provider attempt. The gateway does not spend provider capacity and then ask whether the request should have been allowed.

Output checks have a different constraint. A provider may begin streaming before the final completion is known, but a protected identity cannot receive a byte that the output policy has not examined. When output checks are enabled, the native data plane buffers the winning normalized completion, including tool-call arguments, runs the output chain once, and publishes only after the decision. That delays the first visible byte, but it prevents a partial unsafe response from escaping while the classifier is still deciding. The latency is a consequence of the safety guarantee, not an accidental side effect.

This ordering also explains why guardrail failures are not ordinary provider failures. A provider outage may be eligible for retry or a certified fallback before semantic output commits. A guardrail block, classifier error, or protected-identity fail-closed outcome is a terminal policy decision. Advancing to another provider would not repair the reason the request was blocked; it could instead route the same disallowed or uncertain content around the policy. Treating policy authority as a failover condition would make the boundary meaningless.

The classifier contract is deliberately replaceable. A coarse keyword adapter can support tests, while a production asynchronous HTTP adapter can send a strict inspect contract to a dedicated classifier endpoint. The classifier must not recurse through the public gateway. Protected identities fail closed on timeout, missing adapters, oversized payloads, or other uncertainty; non-protected identities may skip a failed check and continue according to the policy. This difference is an explicit trade-off between availability and enforcement strength, not a hidden property of the model provider.

Privacy places another limit on what a guardrail can become. Logs and durable state keep decision metadata such as policy, identity, capability, action, and latency. Raw prompts, completions, detector payloads, and replacements are not logged or persisted.