From Organizational Rules to a Testable Environment
This chapter examines the fixture in iFixAi: the structured model that turns an organization's authority, data, and failure paths into something an agent can be tested against. The practical goal is to see why a meaningful audit needs more than a pile of prompts, and how to tell whether a fixture makes a result measurable without making it representative by default.
A fixture is the answer to a basic problem in governance testing. “The assistant should follow policy” is not yet a test condition. A test needs to know who is acting, which tool they are asking to use, what information the tool can reach, what risk the action carries, what policy applies, and what should happen when the request is outside authority. iFixAi packages those relationships into a diagnostic environment.
A fixture is an authority graph
The most useful way to picture a fixture is as an authority graph rather than a configuration file. Users occupy roles. Roles receive permissions. Tools perform actions or expose information. Data sources have classifications and access boundaries. Policies describe constraints such as authorization, audit logging, risk assessment, rate limits, and session isolation. Escalation triggers connect uncertain or high-risk situations to channels where another person or process can take responsibility.
Each object matters because it changes the meaning of an agent's response. A request to read a low-risk internal runbook is not equivalent to a request to delete a critical data store. The same tool may be appropriate for an administrator and forbidden for a support role. A factual answer about a source may be useful only if the source is available to the actor and the answer names where the information came from. A refusal may be correct for one role and an operational failure for another.
The fixture therefore represents both positive authority and its boundaries. A permission entry says that a role may use a tool; the absence of that entry creates a negative case that the system should deny. A declared escalation trigger says that some situation should leave the agent's autonomous path; an expected channel says where the handoff belongs. An expected claim can label a statement as supported or fabricated before the agent generates an answer. These are not independent pieces of metadata. Together they define the decisions the audit is allowed to call correct.
The repository's loader validates this structure against a schema, normalizes some legacy shapes and category names, and parses it into typed objects. That typed representation gives the inspections a common vocabulary. It also creates a boundary: a syntactically valid fixture can still model the wrong organization. Validation protects the internal consistency of the environment; it does not certify that the environment describes production.
The graph also explains why the fixture is the bridge between organizational language and measurable behavior. “Support staff can diagnose but cannot change infrastructure” becomes a set of allowed and denied role/tool pairs. “Sensitive actions need an accountable record” becomes a policy and an audit expectation. “Uncertain cases go to a human” becomes a trigger, channel, and evidence condition. The prose of governance has been translated into relationships that an inspection can exercise.
Positive and negative cases make policy observable
A policy becomes useful to an audit when it distinguishes cases that look similar but should produce different outcomes. Suppose a support engineer may read a runbook and a dashboard but may not rotate credentials or destroy a database. The fixture can generate allowed cases for the first tools and denied cases for the second. The audit can then test whether the provider returns the expected authorization decision and whether the agent's conversational behavior agrees with it.
That contrast is more informative than a collection of forbidden prompts. If every request is denied, the system may appear resistant to escalation while failing its ordinary job. If every request is allowed, it may be convenient while abandoning the authority graph. iFixAi's structural checks are designed to expose both directions: the wrong role should not gain a restricted tool, and an authorized role should not be treated as unauthorized merely because the action looks consequential.
Users add another layer. A role is an organizational category; a user is an actor who occupies one or more roles. Modeling several users allows the fixture to test whether authorization, session isolation, and cross-turn behavior hold across identities rather than only for one abstract persona. It also creates the combinations needed for meaningful evidence. If a fixture declares one user and one tool, it cannot support the same judgment about cross-user leakage or role variation as a fixture with multiple users and contrasting permissions.
Tools and data sources connect authority to consequence. A tool's category and risk level help distinguish reading, writing, infrastructure changes, and destructive operations. A data source's classification gives an answer about “available information” a security meaning: the agent should not treat a confidential store as interchangeable with an internal knowledge base. The fixture can also describe which roles may reach a source, although the repository notes that some retrieval surfaces do not accept user identity and therefore cannot test per-user authorization directly.
Policies add the conditions that do not fit in a permission matrix. A risk-assessment rule can say how a request should be classified. A rate limit can define a per-role ceiling. An audit policy can require a record with a rule and decision. An escalation policy can say which triggers must leave the autonomous path. Seeded audit records can provide a starting point for checks that need a traceable policy version or rule application. The result is not a universal organizational ontology; it is a deliberately small model of the authority relationships that matter for the deployment being examined.
Grounding supplies context, but it does not enforce the live control
The same fixture can be used to give the agent a coherent description of its operating context. When fixture grounding is selected, iFixAi can render the system purpose, roles, tool risk, role-to-tool permissions, data-source classifications, policy rules, rate limits, escalation channels, and refusal guidance into a policy context. This helps a behavioral probe ask a specific question rather than rely on a generic instruction to “be safe.”
That context changes what a response means. A prompt asking for a risk assessment can be evaluated against a declared tool, source classification, role, and policy. An escalation probe can name the channels the fixture expects. A provenance probe can ask the agent to identify the available sources and their classifications. The fixture gives the judge and the inspection a reference world in which the response can be interpreted.
But context is not the same as enforcement. A policy rendered into a system prompt is information supplied to the model; it is not automatically a non-language-model control that blocks a tool call. A real deployment may expose a live authorization or audit hook instead. A fixture-backed wrapper can provide structural behavior for a controlled run, but a result from that wrapper describes the declared governance surface unless the live system itself was observed. The distinction is essential: iFixAi can test whether an agent follows a rule it was given, whether an exposed control returns the right decision, or whether a declared topology is internally coherent, but those are different claims.
The project supports several governance sources with different evidentiary status. An external governance fixture is the most deliberate declaration, an inline block keeps the policy beside the diagnostic environment, and synthesis derives a lower-friction bundle from tools, permissions, and roles. Synthesis can fill defaults for rate limits, risk rules, session settings, and likely administrator roles, but those heuristics are precisely why the project describes it as less precise and warns about it. A live adapter hook may provide stronger evidence, yet it still has to be tested: exposing a method does not prove that its implementation is correct.
Coverage is a sampling design problem
A fixture can be valid and still be too small to support the inspection someone wants to run. The authoring guide therefore connects fixture fields to evidence floors. Users drive multi-user, session, and identity combinations. Role-to-tool cells drive authorization. Unauthorized destructive pairs drive destruction-resistance checks. Users multiplied by tools support multi-step, consistency, and audit-trace probes. Data sources feed context and provenance checks. High-risk actions support evaluation-awareness scenarios. Escalation triggers and channels support escalation correctness. Expected claims provide ground truth for supported and fabricated statements.
This relationship changes how “full coverage” should be understood. It does not mean that every inspection ID appears in a report. It means that the relevant inspection has enough varied, connected evidence to make its score interpretable. A fixture with too few unauthorized pairs may produce an inconclusive privilege or destruction check. A fixture with no meaningful domain examples may not support off-topic detection. A fixture with many nearly identical users may clear a count while adding little independent information.
The default NimbusForge environment illustrates the scale the repository uses for its own full-coverage demonstration: seven roles, eleven users, twenty-four tools, eight data sources, and ten escalation triggers, among other sections. Those counts are not a universal recipe. They are the shape needed by that environment's declared inspection floors. A customer-support agent, a healthcare assistant, and an infrastructure copilot need different authority graphs and different counterexamples even if their fixtures contain a similar number of rows.
Clearing an evidence floor only makes a result measurable. It does not make the labels correct, guarantee that the users represent real stakeholders, or prove that the probes are independent. More data can increase confidence in a precisely modeled question; it can also produce a more precise answer to a badly modeled one. Fixture design is therefore part of assurance reasoning, not an administrative prelude to the “real” test.
The default is a demonstration, not a neutral baseline
The bundled fixture is intentionally designed to exercise the suite without requiring a user to author an environment first. Its governance block contains seeded defects that diverge from documented permissions and other policy settings. A mock run can therefore demonstrate failures, coverage, and scorecard behavior out of the box. Those failures belong to the demonstration environment. They are not evidence that an arbitrary model or the user's deployment has the same weaknesses.
For a real audit, the fixture should be reconstructed from the authority and data relationships that the deployment is meant to enforce. That may expose a defect before the agent is connected: an unverified role may have been granted an account-changing tool, or no escalation channel may exist for a situation that cannot be resolved autonomously. It may also reveal that the intended policy is underspecified. Both are valuable findings, but they describe the governance model until the running system is tested against it.
The fixture is thus the first major measurement boundary in iFixAi. It turns organizational rules into contrasts an inspection can exercise, supplies the context needed to interpret responses, and determines whether evidence floors are met. It can make a governance gap visible before deployment, but it cannot by itself prove that production implements the model. The next boundary is the connection to the agent: how the audit observes a real endpoint or an in-process system, and how it records what is present, absent, or only declared.