Why Experience Is Not Enough—and Why Agents Widen the Risk Surface

For professional developers using AI coding assistants, seniority and familiarity with AI are tempting shortcuts for judging security performance. This chapter tests those shortcuts against the study’s cohort result and then widens the frame beyond generated code. The useful judgment is not whether an engineer is “experienced enough,” but whether security-specific knowledge becomes an action at the right moment—and whether the workflow protects the information and system state that an agent can access.

A cohort is a time marker, not a security credential

The USENIX researchers expected a developer’s relationship to AI during professional formation to predict security behavior. That is a reasonable hypothesis. Someone who learned to develop software before assistants were widely available may have stronger habits of independent implementation. Someone who entered the profession with an assistant already available may be more comfortable delegating work and less practiced at reasoning without it. But those are possibilities, not properties guaranteed by a birth cohort of tooling.

The coding sessions did not support a reliable relationship between cohort membership and security performance. The two participants who caught unstated security issues came from different groups and different levels of seniority. P10, an AI-era participant with relatively less professional experience, identified a critical authorization flaw. P12, a more experienced Pre-AI participant, used a deliberate post-implementation question to surface additional security concerns. Participants who missed important issues were distributed across cohorts as well.

The result weakens a common shortcut: years of experience or early exposure to AI can serve as a proxy for secure AI-assisted work. A proxy is useful only when it tracks the capability that matters. In these sessions, the more relevant capability was specific security knowledge applied to a concrete task, together with a habit of asking or checking at the right point. The date when AI entered a person’s professional formation did not reliably tell the researchers whether that bridge would be crossed.

This is a null result in a small, non-random sample, not evidence that experience never matters. More experienced participants tended to articulate more specific security concerns in interviews. That advantage did not consistently appear in their coding-session behavior. The difference suggests that experience may improve a person’s vocabulary or repertoire without guaranteeing that the relevant concern will be retrieved during a fast interaction with an assistant. A developer can know more and still omit the question that the task requires.

The distinction is between capacity and activation. Security knowledge is a capacity: knowing that authorization, input handling, secret management, or concurrency can create risk. Activation is the act that brings the capacity into the workflow: adding a condition to the prompt, challenging an assumption, running a targeted test, or refusing an action that crosses a boundary. The study’s observation is that activation varied in ways the cohort labels did not explain reliably.

That finding changes how teams should interpret experience.