Trust, Review, and the Illusion of Equivalence

For developers using AI coding assistants, saying “I do not fully trust the output” is not yet a review strategy. The specific problem is calibration: does the scrutiny applied to a suggestion match the security stakes of the code, or does every plausible answer receive roughly the same treatment? This chapter examines the relationship among developer trust, assistant confidence, and review depth so readers can distinguish conditional reliance from a vague posture of skepticism.

Skepticism is not a review policy

The participants in the USENIX study generally did not describe AI-generated code as fully trustworthy for security-sensitive implementation. Their reported trust ratings clustered between 4 and 7 on a 10-point scale. That range communicates a sensible intuition: the assistant is useful, but its output requires checking. A developer who assigns a middle-range score is not necessarily careless. They may be expressing an appropriate willingness to use the tool while reserving final responsibility for themselves.

The difficulty is that a rating does not specify what happens next. Trust in an assistant is not one decision applied to an entire codebase. It is a series of conditional decisions. A developer might trust the tool to suggest boilerplate, explain an unfamiliar library, or outline a routine function, while requiring independent verification for authorization logic or code that handles sensitive data. The useful question is not “Do I trust the assistant?” but “What may this assistant decide, what must I verify, and what evidence would change my decision?”

That distinction separates conditional reliance from wholesale acceptance. Conditional reliance gives the assistant a bounded role and assigns a corresponding verification step to the developer. If the assistant is asked to draft a routine transformation, the developer may inspect the interface and run tests. If it is asked to implement a profile update, the developer must also inspect who is allowed to update which record and whether caller-controlled values are being treated as authority. The code’s apparent complexity is not the only factor. The consequence of a wrong assumption matters.

Yet the study found that practical trust was not reliably calibrated to security sensitivity. Participants applied similar levels of trust to boilerplate and security-critical code, even though the assistant provided no indication that the two categories carried different risks. This is an interface problem as well as a human problem. A single polished suggestion visually presents routine scaffolding and an access-control decision as comparable units of work. If the tool does not expose the difference, the developer must infer it from the surrounding task and supply all of the risk classification themselves.

The “junior colleague” mental model helps explain why people can be skeptical and still use the assistant extensively. Several participants described the tool as useful but not authoritative: like a less experienced engineer whose work must be checked. This model preserves human ownership.