Overview

A working scope test moves an AI report from observed behavior to a defensible response by connecting five elements: behavior and preconditions, security property or material harm, contextual impact, evidence and reproducibility, and response route. It handles authorization failures, prompt injection, and harmful output without treating labels as verdicts. The test keeps uncertainty visible, distinguishes insufficient evidence from no vulnerability, and treats direct patchability as a response question rather than an eligibility gate. Its value depends on a public policy that makes permissions, channels, evidence requirements, exceptions, alternate routes, and commitments observable.

Learning outcomes

A sound AI scope decision connects behavior and preconditions to property, contextual impact, evidence, and a response route.
Prompt injection, authorization failures, and harmful output need different judgments because impact depends on authority, affected state, and downstream use.
Uncertain or non-patchable findings should remain actionable through evidence requests, alternate routes, mitigation, monitoring, or coordination.

Keywords

AIscopeevidenceauthorizationreproducibilitytriage

A Working Test for AI Vulnerability Scope

The central judgment now is whether an AI report has enough structure to move from an observed behavior to a defensible response. The useful test connects five things: what happened, what property or harm is implicated, what context makes the effect possible, what evidence supports the claim, and which route can reduce the risk. It is not a mechanical classifier. It is a disciplined way to keep category labels, uncertainty, and organizational responsibility from being confused.

That distinction matters because the preceding evidence points in two directions at once. Vendors accept security-goal categories more consistently than model or content labels, yet AI mechanisms can create effects that ordinary policy wording does not explain well. A working test must preserve both observations. It should make a clear authorization failure easy to recognize, give a prompt-injection report a way to demonstrate its real impact, and keep a harmful-output concern from disappearing merely because it does not fit a bounty queue.

Five questions connect behavior to a scope decision

Start with the behavior and its preconditions. What did the system do, and what had to be true for it to do it? A useful description includes the relevant input or sequence, the model or feature involved, the account or permission state, the connected data or tool, and the conditions under which the behavior appeared. This is not a demand for exhaustive implementation detail. It is the minimum context needed to distinguish an accidental oddity from a condition another person could use.

Next ask what security property or material harm is implicated. The property might be confidentiality, authorization, integrity, availability, control over a protected resource, or another explicit or implicit rule in the product. If the concern is safety or abuse rather than conventional security, name the affected people, exposure, or downstream consequence instead of assuming that the category speaks for itself. This question translates “the model did something surprising” into “which boundary may have been crossed?”

The third question is contextual impact. Who could be affected, and through what path? Does the behavior remain inside the researcher’s session, or can it affect another customer, a stored record, a tool invocation, a decision, or a deployed workflow? Does the same model response have different significance when it is displayed to a user than when it is trusted by an automated system? Context is not an excuse for arbitrary scope. It is the information that tells the vendor whether the alleged property is actually at stake.

The fourth question concerns evidence and reproducibility. What was observed directly, what was inferred, and what remains uncertain? Can the vendor reproduce the effect, or can the researcher describe a probabilistic condition with enough detail to investigate it? A report that includes the model version or deployment context, the relevant sequence, the observed output or action, the affected asset, and the limits of the test gives triage something concrete to validate. A single dramatic output without an impact path may not.

The fifth question is response route. If the claim establishes a security vulnerability, should it enter the ordinary disclosure or bounty process? If it is a safety or abuse concern, is there a dedicated channel with a real owner?

Published by DeepMox

FAQ

What are the five questions in an AI vulnerability scope test?

The test asks what happened and its preconditions, what security property or harm is involved, who and what context are affected, what evidence supports the claim, and which response route can reduce risk.

Can a vendor reject an AI report because it is not reproducible?

It may need more evidence to validate and prioritize the report, but intermittent or probabilistic behavior is not automatically irrelevant; the policy should state what context and evidence can support further investigation.

How should a prompt-injection report show security impact?

It should show what authority the model had, which data or state it could reach or change, who could be affected, and whether the effect was demonstrated.

Previous9 minAI Findings: Scope and Exclusions

Next9 minAI Bug Bounty Policy Design