When AI Writes the Code, Where Does Security Go?
This series examines professional developers’ security practice when AI coding assistants take part in implementation. The central question is not simply whether an assistant can generate vulnerable code. It is where security judgment enters the workflow, what happens when it enters too late, and how teams can make that judgment more reliable. The evidence points to a consequential shift: AI assistants often move security thinking from the act of specifying and writing code to the act of reviewing it afterward. Understanding that shift gives readers a better way to judge both the promise and the limits of AI-assisted development.
The security problem is not only bad code
AI coding assistants have become part of everyday software work because they compress activities that once consumed substantial time. Developers can ask for a feature, a project outline, a utility function, or an explanation of an unfamiliar library, then revise the result through a conversation. The gain is most obvious in routine or well-scoped work: boilerplate appears quickly, a new project becomes runnable sooner, and unfamiliar territory becomes easier to explore.
Security does not disappear when this happens. It changes location.
When a developer writes a function directly, security can be part of the original reasoning. They may decide who is allowed to perform an action, which input is untrusted, what data must remain private, or what failure should be recorded before they write the implementation. Those decisions shape the code from the beginning. With an assistant, the first interaction often has a different shape: “Create this endpoint,” “Initialize this backend,” or “Fix this function.” The request describes what the software should do. Unless security constraints are also made explicit, the assistant has been given a functional target but not necessarily the conditions under which that target is safe.
That distinction matters because software can be functionally convincing while being security-incomplete. An endpoint can accept the right fields, update a database, and return a success response while failing to verify whether the requester is allowed to change the named account. A project can start successfully while storing secrets in an unsafe way. A debugging fix can remove an error while leaving a race condition that lets one user’s data cross into another user’s path. Correct behavior on the happy path is not the same as correct behavior at a security boundary.
The USENIX study by Faisal Haque Bappy and colleagues treats this change in timing as its primary subject. It asks how professional software engineers think about and practice security when AI assistance is available, rather than measuring only the vulnerability rate of generated output. Its phrase “from preventive to reactive” describes a change in when security is applied. Preventive security is considered while requirements, design, and implementation choices are being formed. Reactive security is applied later, when someone reviews, tests, or responds to code that has already been generated.
“Reactive” does not mean that every review is useless, nor that developers no longer know security principles. It means that the workflow makes security a downstream obligation. That obligation must compete with the satisfaction of seeing a feature work, with delivery pressure, and with the confidence created by an answer that looks complete. The important question is therefore not whether developers care about security in the abstract. It is whether the interaction helps them carry that concern into the request, the generated design, and the decision to accept the result.
Prevention and reaction happen at different moments
A simple way to see the difference is to follow a request through the development process. The request first crosses from the developer’s intention into the assistant’s interpretation. The assistant then returns code that appears to satisfy the stated function. Only afterward does the developer decide how deeply to inspect the assumptions inside that code.
If the security requirement is present at the beginning, it can constrain the assistant’s interpretation of the task. It can tell the system that identity must come from a trusted session rather than from user-controlled input, that sensitive values must not be exposed, or that input handling must account for malicious data. Those instructions do not guarantee secure output, but they make security part of what “correct” means before code is produced.
If the requirement is absent, the generated code may still be useful. The problem is that the burden of adding the missing condition now rests on review. The developer has to notice that the original request was incomplete, recognize which security properties matter, inspect the relevant assumptions, and decide whether the assistant’s implementation actually satisfies them. Review is essential in either path, but the second path asks review to reconstruct decisions that preventive reasoning would have established earlier.
This is the core tension in the study. Security awareness can exist as a belief, a memory, or an answer to a direct interview question without becoming part of the initial AI interaction. A developer may know that authorization matters and still ask only for a working profile endpoint. They may understand that secrets require special handling and still accept a project scaffold before checking its configuration defaults. The distance between knowing a rule and invoking it at the right moment is where the workflow becomes vulnerable.
The distinction also explains why a study of real interactions adds something to benchmark results. A benchmark can reveal that a generated function contains an unsafe pattern. It cannot by itself show whether the developer asked for a security property, noticed the pattern, trusted a passing test, or treated the assistant as a collaborator whose work required inspection. Those behaviors determine whether a flaw is corrected before release or becomes part of the accepted implementation.
NIST’s Secure Software Development Framework, or SSDF, provides a useful lifecycle lens for this problem. Its four broad groups—preparing the organization, protecting the software, producing well-secured software, and responding to vulnerabilities—place security across requirements, design, development, review, testing, release, and learning from failures. SSDF is not an AI-specific experiment or a prescribed prompt template. Its value here is conceptual: it makes clear that code review is one control in a larger system, not the point at which security begins and ends.
What the study is actually trying to reveal
The USENIX researchers combined two forms of evidence in a single session lasting roughly 45 to 60 minutes. They first conducted semi-structured interviews about participants’ everyday security practices, use of AI tools, and confidence in finding issues before code shipped. They then observed a think-aloud coding task in which participants used Gemini CLI with Gemini 2.5 Pro. The researchers watched prompts, iterations, code changes, decisions, and omissions rather than relying only on what participants said they usually did.
The study involved 15 professional software developers. The participants were grouped according to when AI tools entered their professional formation: Pre-AI developers began before such tools were widely available, AI-era developers encountered them as they entered the workforce, and AI-native developers trained and worked during the period when assistants were already accessible. Fourteen participants completed the coding task because one interview participant declined that part of the session. The tasks covered three ordinary development situations: implementing a feature, initializing a project, or debugging existing code.
The task prompts intentionally contained no explicit security requirements. That choice is important. If the prompt had said “make this secure,” the study would have measured how people respond to a security cue. By leaving security unstated, the researchers could observe whether participants introduced it themselves while pursuing a functional goal. The design therefore focuses on the boundary between a developer’s existing awareness and the behavior that an AI-mediated workflow elicits.
The result should be read as exploratory evidence, not as a universal measurement of developers. The sample is small and male-skewed, and participants ranged from entry-level to mid-level roles. One AI setup cannot stand in for every coding assistant or agent configuration. The interview came before the task and may have prompted more reflection than ordinary work would, while the think-aloud setting may have encouraged deliberateness. These limitations reduce the study’s ability to estimate how common a behavior is. They do not make the observed mismatch irrelevant: even after discussing security, most participants still did not put security requirements into their initial task prompts.
The broader judgment this evidence supports is measured but significant. AI assistance does not automatically erase security knowledge, and it does not inevitably make every developer less careful. It changes the division of labor. The assistant makes functional production easier; the developer remains responsible for identifying the security properties that the functional request did not state and for deciding whether the result deserves trust. When that responsibility is left implicit, security can become something to catch after the fact rather than something that shapes the implementation from its first description.
That is why the subject of this series is not “AI code” in isolation. It is the relationship among a developer’s knowledge, the assistant’s interaction model, the generated or modified software, and the controls surrounding the workflow. The next question is how the study separated those parts—what developers reported, what they prompted, what they reviewed, and what the assistant produced—and what confidence each kind of evidence deserves.