The Scope Problem: When Does an AI Behavior Become a Vulnerability?
This series examines how AI vendors define the boundaries of bug-bounty and responsible-disclosure programs, and why those boundaries are becoming harder to defend with a simple list of allowed and forbidden findings. The immediate focus is not how to break an AI system. It is how a vendor, researcher, or coordinator decides what a reported behavior means, which channel should receive it, and what response would actually reduce risk. The practical goal is a clearer way to distinguish an AI vulnerability from an ordinary limitation, an unsafe product behavior, or an incident that needs a different kind of attention.
That distinction matters because AI systems make surprising behavior easy to produce, but surprise is not the same as security impact. A model can give an inaccurate answer without giving an attacker access to another customer’s data. A prompt can bypass a safety instruction without changing any account, system state, or protected information. Conversely, a seemingly ordinary conversational feature can become a serious vulnerability if it can be induced to disclose secrets or take an unauthorized action. The words used to describe the behavior are clues. They are not the judgment itself.
The policy question is becoming urgent. A recent study of public vulnerability-disclosure policies from 264 AI vendors found that 36% had no public reporting or disclosure channel, while only 18% explicitly addressed AI security. The same study found a wide divide between vendors that proactively clarify AI findings, vendors that remain silent, and vendors that restrict or reject them. For a researcher, this is not merely a question of whether a bounty will be paid. An unclear boundary affects whether testing is authorized, what evidence is worth collecting, whom to contact, and whether a report will be treated as useful information or dismissed as a product complaint.
A better scope definition must therefore do more than name AI products. It must connect an observed behavior to a security property or material harm, preserve the context and mechanism that make the behavior possible, and identify a plausible route from report to mitigation. That is the central problem this series develops.
The policy gap is a scope problem
The public-policy evidence shows why the subject cannot be reduced to a debate over terminology. In the USENIX study, 264 companies were selected as AI vendors because their products had substantial AI capabilities, including foundation-model providers, generative-AI applications, and AI or machine-learning infrastructure. The sample was not a census of every company that uses a model, and the findings describe public documents rather than private security practice. Even with those limits, the pattern is consequential: more than a third of the sampled vendors offered no visible channel for vulnerability reporting, and fewer than one in five explicitly mentioned AI issues.
The absence of a sentence about AI does not prove that a vendor rejects AI reports. Some silent vendors may accept a finding under a general security policy. Others may patch an issue in practice without explaining the route to researchers. But the researcher cannot reliably tell which situation applies. Silence transfers the cost of interpretation to the person who is testing the system. They must infer whether a prompt injection is a security report, whether a model extraction attempt is permitted, whether a harmful output belongs in a bounty program, and whether the receiving organization can change the relevant component.
The study’s three profiles make this uncertainty more concrete. Proactive vendors clarify which AI assets and behaviors are eligible and sometimes provide dedicated channels, severity guidance, testing material, or examples. Silent vendors have not updated their public rules to address AI, even though some may still receive relevant reports. Restrictive vendors either lack a discoverable disclosure channel or explicitly exclude some AI reports. These are positions expressed in public policy; they are not a ranking of the vendors’ overall security capability. Still, they create very different expectations for someone deciding whether and how to report.
The distinction between a visible scope and an actual internal practice is important. A policy is an interface between an organization and the people who supply it with security information. If the interface is missing, vague, or routed through a general customer-support form, the organization may still have strong internal controls, but external researchers have no dependable way to contribute. A usable policy lowers the uncertainty around permission, evidence, communication, and escalation before a dispute begins.
This is why “AI bug” is too broad to function as a useful category. It could refer to unauthorized data access, model extraction, an unsafe generated response, an unreliable classifier, a compromised plugin, a supply-chain dependency, or a user trying to elicit prohibited content. Those behaviors do not share one impact, one owner, or one remediation route. A scope definition that treats them as one class either becomes so broad that triage loses meaning or so narrow that important failures disappear between categories.
Vulnerability is not the same as surprising behavior
The CERT Guide to Coordinated Vulnerability Disclosure offers a useful baseline. It defines a vulnerability as a set of conditions or behaviors that allows an explicit or implicit security policy to be violated. The condition may come from a defect, a design decision, an unexpected interaction, a configuration, or an environmental change. In a more operational formulation, it is an undesirable, exploitable, and likely unintended feature that enables an attacker to perform an action that would otherwise be unavailable to them.
That definition does not require a particular implementation mechanism. It also separates several terms that are often blurred in AI discussions. An exploit is software or a method that uses a vulnerability. An incident is a violation or attempted violation of a security policy, and it may involve a vulnerability, an exploit, both, or neither. A model’s inaccurate answer may be a quality failure without being an exploit. A harmful output may be a safety concern without giving an attacker an unauthorized capability. But if a model or its surrounding application can be induced to reveal protected data, change a victim’s account, or cross an authorization boundary, the relevant question is the violated property—not whether the input looked like a traditional injection.
The AI-specific mechanism still matters. AI systems learn probabilistic behavior from data and tuning rather than following only deterministic instructions written by developers. The mechanism can affect reproducibility, testing permission, evidence quality, and the choice of fix. A prompt injection may exploit the way instructions are prioritized. A jailbreak may bypass a safety boundary. Model extraction may target proprietary weights or behavior. A hallucination may become security-relevant only when another component trusts the output and uses it to make a consequential decision. These mechanisms should be described accurately, but their names cannot substitute for an impact analysis.
The outcome-oriented view and the mechanism-oriented view are therefore not rivals that require choosing one and discarding the other. The outcome tells us why the behavior matters. The mechanism tells us how it occurs, under what conditions it can be reproduced, and what kind of response might work. A report that says only “the model can be jailbroken” leaves too much unresolved. A stronger report explains what the bypass permits, who can trigger it, whether it affects only the requester’s session or crosses into another person’s data or state, and what evidence demonstrates the claim.
Coordinated disclosure also makes scope a process question. The CERT guide describes CVD as a continuing sequence of discovery, reporting, validation and triage, remediation, public awareness, and deployment. It is not completed merely because a report was submitted, a bounty was paid, a patch was released, or an advisory was published. Participants repeatedly ask what action should follow from their knowledge and who else needs to know what, and when. The process ends only when there is no remaining action or recipient that can reduce the risk.
That broader view matters for AI because “accepted” and “fixed” are not interchangeable. A vendor may accept a report about a model behavior but respond with a guardrail change, an access-control change, a configuration recommendation, a feature restriction, monitoring, or coordination with a model provider. Another vendor may reject a bounty while still directing the finding to a safety or abuse team. The first decision concerns whether the signal deserves structured attention. The second concerns how risk can be reduced.
From labels to a response path
The most useful starting point for a new report is not “Which AI attack name applies?” but “What changed, for whom, and against which boundary?” That question does not make every issue a vulnerability. It makes the decision inspectable.
First, identify the behavior and its preconditions. What input, sequence, account state, tool connection, model configuration, or downstream workflow is required? Is the effect repeatable, probabilistic, or dependent on a particular deployment? A one-off surprising answer and a reliable path to another customer’s information should not arrive at triage as equivalent claims.
Second, identify the property or harm. Does the behavior violate confidentiality, authorization, integrity, availability, or control over a protected resource? Does it expose people to a material safety or abuse risk that the security queue is not designed to handle? The answer may be uncertain at first, but the report should make the uncertainty visible rather than hide it behind a label.
Third, connect the evidence to a response route. A clear security violation may belong in the ordinary vulnerability program. A safety issue that does not meet the program’s vulnerability definition may still warrant a dedicated abuse or safety channel, monitoring, or escalation. If the receiving vendor does not control the underlying model, it should relay the report or clearly identify the model provider’s path rather than leaving the researcher to guess.
This approach also explains why public scope language matters even when generic terms might technically cover an AI finding. A researcher needs to know whether testing is authorized and what conduct is prohibited. They need a secure channel, an acknowledgement, and a reasonable expectation of communication. They need to know what evidence helps triage and whether a report can be appealed when the vendor says the behavior is expected, non-reproducible, or not directly patchable. The policy does not need to enumerate every future attack. It does need to explain the reasoning by which unfamiliar reports will be judged.
The CERT guide calls disclosure a problem of harm reduction rather than a search for a perfect formula. That is a useful standard for AI policy. A vendor should ask whether its wording lowers the period of exposure, encourages high-quality reports, protects researchers who follow the rules, and helps deployers or users take action. The study of AI policies suggests that many vendors are still deciding whether model behavior belongs inside traditional security programs, outside them, or in a parallel channel. The answer should not be hidden in a checklist.
A defensible scope boundary can be explicit without pretending to be automatic: an AI behavior is a vulnerability when there is sufficient evidence that a condition in the AI-enabled system can be used to violate a security property or cause material, context-specific harm beyond intended authority. The policy should then state the relevant context, testing permissions, evidence requirements, exceptions, ownership path, and possible remediation routes. Some findings will remain cases for judgment. That is not a failure of definition; it is an honest acknowledgment that AI systems operate across probabilistic behavior, human use, model dependencies, and changing environments.
The central question is consequently not whether AI is “special” in the abstract. It is whether the disclosure system can recognize what the AI behavior changes, route that information to someone who can act, and reduce harm without making researchers guess at the boundary. A label can begin that conversation. Only a clear property, context, evidence trail, and response path can finish it.