The Boundary Between an Agent and Its Auditor
This chapter examines the provider boundary in iFixAi: the interface through which the diagnostic talks to an AI agent and, when available, observes the controls around it. The key judgment is not whether a provider implements many methods, but whether each exposed surface lets the audit make a specific, defensible claim about the agent's behavior or governance.
A fixture says what should be true. The provider is how iFixAi finds out what the system can do, what it reports, and what it refuses to expose. That separation lets the project work with a deployed HTTP agent, a model API, an in-process application, or a custom adapter without pretending that all of them provide the same evidence.
One required path, many optional surfaces
The provider abstraction has one essential behavioral operation: send a conversation to the system and receive its response. That is enough to connect a custom agent to the basic prompt-driven part of the diagnostic. An adapter can translate iFixAi's message objects into the agent's native request format and return the resulting text without requiring the agent to adopt iFixAi's internal architecture.
Everything deeper is optional and typed. A provider may expose tools and invoke or authorize them; return retrieved sources; provide an audit trail; report a routing decision; expose its governance architecture; apply an override; identify the configuration version; or report confidence and human-fallback decisions. More specialized surfaces cover outcome reconciliation, deployment and confirmation gates, standing-authority revalidation, training-corpus screening, model identity, and delegation scope. These are not interchangeable status flags. Each one gives an inspection a different question to ask.
That design creates two related paths through the system. The behavioral path sends a prompt or conversation to the agent and examines the response. The structural path calls a typed capability directly and compares its result with the fixture's expectation. A retrieval hook can return which sources were found; an authorization hook can say whether a role may invoke a tool; an audit hook can return the record made by a decision. The provider owns the translation from the deployment's actual machinery into these typed results. The inspection owns the comparison with the declared governance model.
The boundary is deliberately thin. iFixAi does not require every agent to be rebuilt as a plugin or to use a particular orchestration framework. The documentation recommends an HTTP endpoint for a deployed system and a ChatProvider subclass for an in-process agent. In both cases, the adapter is responsible for preserving the meaning of the deployment's behavior. If a custom adapter turns every failure into an empty string, or reports a policy decision without the policy evidence that supports it, the interface may be satisfied while the audit is weakened.
That is why portability and observability are separate virtues. A single message method gives broad reach. Optional hooks give depth. The absence of depth should be visible rather than hidden behind a universal claim that the agent was fully audited.
The diagram's important distinction is between a live control plane and a declared governance surface. Both can be made visible through the provider boundary, but they do not support the same strength of conclusion.
Capability detection answers presence, not correctness
Before an inspection relies on a structural method, iFixAi checks what the provider appears to expose. The capability detector probes representative hooks and records whether they return a result. If a provider can list tools, retrieve sources, return an audit trail, authorize a tool, or report configuration, the corresponding capability becomes available to the runner. If the base method returns no result, the capability is treated as absent.
This is useful, but it is easy to overread. A positive capability result means that an interface answered a probe. It does not mean that authorization is correct, that the audit trail is complete, or that the confidence estimate is calibrated. Those are inspection questions. Presence is the precondition for measurement; it is not the measurement itself.
The distinction resembles a laboratory instrument. Detecting that a scale turns on does not establish that it weighs accurately. In iFixAi, capability detection tells the runner whether it can ask a typed question. The inspection then supplies roles, tools, sources, policies, or test conditions and evaluates the returned answer. A provider that exposes authorize_tool but allows a forbidden role to invoke a destructive tool has a measurable failure, not a successful audit simply because the method exists.
A useful consequence follows from this two-stage design. The inspection can ask the same provider surface questions that are deliberately outside the ordinary user conversation. It can use an unknown role or tool as a holdout, request an audit record for a fresh action, or compare a declared policy version with the version returned by the control plane. These probes do not ask the model to describe its own safeguards; they ask the system that owns the decision to return a typed result. If the returned object says a call was denied but also says the tool executed, the contradiction is evidence about the control boundary. If it gives no policy rule or trace, the denial may be operationally safer than an allow but still fail the audit's accountability condition.
The behavioral path has a different ownership problem. The agent owns the wording of the response, but the judge must decide whether that wording demonstrates the expected outcome. A provider adapter should not pre-classify every answer as safe merely because it contains the word “refuse.” Nor should the adapter turn every short response into a failure. The inspection and its rubric define what counts as a concrete behavioral success, while the provider preserves the original response as evidence. The boundary is therefore a contract about meaning, not only a transport format.
This is also why a custom adapter is an engineering decision rather than a clerical one. Exposing only send_message makes the system reachable and allows behavioral checks, but it leaves the auditor unable to verify control-plane properties directly. Exposing a hook creates coverage, but it also commits the adapter to accurate identity, state, error, and provenance semantics. The more authority an agent has, the more consequential that commitment becomes: a missing audit hook around a read-only assistant is a coverage gap, while the same gap around destructive infrastructure actions may prevent the run from establishing a mandatory control.
The opposite case is just as important. If an agent has no exposed audit trail, iFixAi should not infer from a polished response that a record exists. If it has no confidence or human-routing hook, the corresponding structural question may be unanswerable. The report can say that evidence is insufficient. That is a limitation of the observed boundary, not a fabricated score about an invisible subsystem.
The provider boundary also defines ownership. The live agent or its control plane owns the actual decision and state. The adapter translates that state into an interface. The inspection owns the expected comparison. The runner owns the status and report. Keeping those roles separate prevents an adapter from quietly changing a failure into a pass merely to fit the reporting schema.
Declared, synthesized, and live governance are different evidence
A fixture can carry a governance block that supplies the structural policy surface for a run. iFixAi can bind that declaration to a provider through a governance wrapper, allowing structural inspections to receive typed authorization, audit, version, override, risk, session, and related responses. This is useful for testing a policy model and for exercising the diagnostic without a live control plane.
But the wrapper does not turn a declaration into production enforcement. It makes the declared rules executable as a test surface. If the declaration grants too much authority, the run can expose that defect. If the declaration says that audit logging exists, the wrapper can make an audit-shaped result available. Neither outcome proves that the deployed service uses the same rules or records the same events unless the live service itself is the provider being observed.
The project also supports governance synthesis: deriving a lower-friction policy bundle from diagnostic tools, permissions, and roles when an explicit governance block is absent. Synthesis can supply conservative defaults for items such as rate limits, risk rules, session settings, and likely override roles. That helps a first run produce useful structural checks, but the heuristics are not the same as an organization's authoritative policy. The project describes this path as less precise and warns that it was synthesized.
A live hook has a different failure mode. It may expose real control-plane data but implement the contract incorrectly, return stale state, omit important events, or disagree with another surface. That is why the presence of live hooks does not end the audit. Cross-hook consistency checks and the inspections themselves still test whether the observed controls behave coherently. Live evidence is stronger than a declaration for some claims, but it is not automatically correct.
A responsible scorecard should therefore preserve the source of each control result. “Authorization passed” is incomplete. The reader also needs to know whether the decision came from a live endpoint, a provider adapter backed by a declaration, or a synthesized policy. The method changes the claim boundary even when the visible pass/fail label is the same.
Missing evidence is not a guessed failure
The engine distinguishes four broad outcomes: pass, fail, inconclusive, and error. An absent hook generally produces no structural evidence, which can make an inspection insufficient rather than silently failing. A fixture with too few relevant cases has a similar problem. The result says that the run did not establish the property, not that the property was violated.
That distinction does not make every missing result harmless. Mandatory inspections fail closed when their evidence is insufficient, because an organization should not receive a clean headline grade after skipping a critical gate. The project also defines genuine not-applicable cases, such as a tool-less target for the tool-governance check or a fixture with no destructive capability for destruction resistance. Those cases are different from a declared destructive capability whose authorization surface could not be reached.
Errors carry a different meaning again. A configuration or runner failure means the inspection did not execute correctly. A provider or judge communication failure may make individual evidence unscorable rather than becoming a behavioral failure. A malformed judge response is not automatically evidence that the agent failed its task. The report's status and warning text are part of the measurement, because they explain whether a score is a measured behavior, a failure of the behavior, or a failure of the measurement path.
For a reader, this creates a useful rule: before asking whether an agent passed, ask whether the relevant surface was available, whether enough evidence was produced, and whether the status represents behavior or infrastructure. A score without that context is less informative than an explicit gap.
Structural authorization and conversational resistance are different
The provider boundary makes it possible to compare two questions that are often collapsed. A structural authorization inspection can ask whether a role/tool pair is allowed at the API boundary. A conversational inspection can ask whether the agent resists a three-turn attempt to talk its way into the same restricted capability. The first tests enforcement where action is authorized. The second tests whether the agent preserves the rule in language under pressure.
A deployment can pass one and fail the other. A robust API gate may prevent execution even if the model promises to comply. A persuasive model may verbally refuse most requests while a weak control plane still allows a caller to invoke the tool directly. Treating those as one score hides where the defense actually resides. iFixAi keeps them separate because governance is both a property of the control boundary and a property of the agent's behavior around that boundary.
The same logic applies to retrieval, auditability, confidence, and escalation. A provider hook can tell the auditor what the system exposes; a prompt-based probe can test what the agent says or does when the hook's information is relevant. The adapter is the boundary that lets these questions coexist without pretending they are the same measurement.
A fixture can therefore make the audit precise, and a provider can make the audit observable, but neither can eliminate the need to interpret evidence. The next step is to connect the fixture's expectations and the provider's surfaces into one end-to-end model: what exactly does a green result describe when the control is live, when it is declared, and when the agent merely gives a convincing answer?