What Counts as Evidence at the Finish Line
A completed-looking result is not yet a traceable result. For AutoResearch, the decisive question is whether a reader can connect a conclusion to the route, model identity, experiment stage, artifact set, and review that produced it. This chapter follows those connections from provider configuration through execution receipts and independent challenge. The goal is a practical reading rule: understand what the records prove, what they only make inspectable, and where they stop short of scientific validity.
The route is part of the result
A model name is not a complete description of a model call. In AutoResearch, a role selects an ordered list of model aliases. An alias selects one or more routes. A route selects an endpoint, a wire-level model name, and request parameters. The endpoint selects the protocol dialect, transport behavior, URL source, and credential variables. Two aliases can look different while reaching the same underlying identity, and two aliases with similar names can use different routes. The path matters because the response depends on more than the label printed in a prompt.
The project’s unified provider configuration makes these relationships explicit. Private values such as URLs and keys belong in the environment; the checked-in configuration describes their variable names and the relationships among roles, aliases, routes, and endpoints. A normal role tries its configured candidates in order. A panel role uses all of its candidates. The same effective configuration is intended to drive the Python pipeline, live preflight, the official Claude Code projection, and the MCP reviewer and critic paths.
That shared resolver addresses a reproducibility problem that is easy to underestimate. If preflight checks one fallback chain while the runtime uses another, a green preflight proves very little. If a reviewer tool has its own hard-coded provider assumptions, the code that was tested may not be the code that actually judged the result. AutoResearch therefore keeps the route definition and the role definition separate but makes all consumers resolve them from one effective configuration.
The model that actually answered is also retained. This is different from recording the first candidate in a list. A role may fall back after a route fails, and the fallback may have a different identity. For a single-seat role, that changes the provenance of the decision. For an independent panel, counting the configured aliases rather than the identities that actually responded could create the appearance of plurality without the corresponding evidence.
The configuration layer does not make every endpoint interchangeable. It can represent supported request dialects and route parameters, but it cannot repair a gateway that rejects the dialect, a model that lacks the required tool behavior, or credentials that are not available at runtime. Live preflight sends minimal requests to discover some of those failures. Even a successful probe establishes reachability and shape, not the quality of the model’s later reasoning.
Failure types change what may advance
AutoResearch’s provider layer avoids reducing every unsuccessful call to “the model failed.” A response can be delivered with usable text, returned successfully but empty, returned in an unreadable shape, rejected by the endpoint, throttled, or met with a remote error. The request may also be unreachable, unconfigured, or geo-blocked. These states lead to different actions because they describe different causes.
An empty response is not the same as an unreachable service. The endpoint may have returned a valid success envelope while the model spent its budget on hidden reasoning or produced no text. An unreadable response is different again: the endpoint answered, but the client could not find the promised structure.