Seeing Inside a Black Box Without Its Source
H3Act is the central subject here: a knowledge-driven system for measuring HTTP/3-to-HTTP/1.1 semantic conversion anomalies in black-box content delivery networks. The focus is not on treating a language model as an oracle, but on understanding how generated cases, edge behavior, and origin-side records become evidence. That distinction gives readers a practical way to judge an automated lead, a confirmed anomaly, and a claim about real security impact.
A CDN's most important code is usually invisible to an outside tester. The tester can send a request and observe a response, but cannot normally see the edge's routing decision, the translation step, the cache key, or the upstream request. Traditional fuzzing methods often depend on code coverage or on a detailed grammar written in advance. When those signals are missing, random variation can produce many inputs that are rejected immediately and few that reach a meaningful conversion path.
H3Act addresses that problem by treating the CDN as a system with two observable ends. It asks what the client sent to the edge and what the origin received after the edge made its decisions. The difference between those observations is not automatically a vulnerability. It is a lead that must be tied to a protocol rule, a specific downstream behavior, and a consequence that the evidence supports.
A CDN that reveals only its edges
The study's threat model gives the black box a concrete shape. The tester can act as a legitimate HTTP/3 client and send requests to a CDN ingress node. The tester cannot directly access the target origin, the CDN's internal network, or its source code. Instead, the experimental setup uses a server controlled by the researchers and configured behind the CDN. That controlled origin records the HTTP/1.1 requests and responses that arrive after the CDN has decided to forward them.
This arrangement is important because it separates reachability from observation. The tester does not need to see inside the edge if the origin can report the representation it received. At the same time, the controlled origin limits what the experiment can claim. It can reveal how the CDN translates and forwards a constructed request under a particular configuration. It cannot establish how an undisclosed production origin, cache policy, authentication layer, or customer deployment would react in every circumstance.
The edge itself has several responsibilities that interact. It terminates the client-facing HTTP/3 connection. It may serve a cached response, decide that a representation must be fetched or revalidated, select a back-to-origin route, and construct an HTTP/1.1 request. Those decisions may happen at different moments and may be made by different components. The origin log is therefore a view of the final handoff, not a complete trace of every internal decision.
That distinction explains why ordinary request-response testing can miss semantic conversion failures. A cache hit may return a normal response without exercising the origin path at all. A cache miss may cause the edge to request more data than the client asked for. A malformed value may be rejected by one component but accepted by another before it reaches the origin. A client-visible status code does not say which of these paths occurred.
H3Act's response is to make the test case meaningful before it is sent. The framework draws on protocol specifications and prior security research rather than mutating bytes without a model of their purpose. It generates cases that are intended to be valid HTTP/3 messages, then uses the controlled origin and the client-side observations to ask whether the CDN preserved the request's meaning. The method is differential in the broad sense: it compares two representations and their effects rather than relying on one response in isolation.
The comparison also changes what “feedback” means. In a conventional fuzzer, feedback often means a new code path or a crash signal. In this setting, the useful signal may be that a request constraint or identity changed, that an origin received unexpected bytes, or that the timing of forwarding created an unusual resource commitment. H3Act's analyzer turns such observations into guidance for later cases. The goal is not simply to generate more traffic; it is to refine the question being asked of the translation boundary.
The researchers kept the experiment controlled and bounded. The paper describes a client-side machine running H3Act and a separate origin machine, with the origin configured to record full request and response headers and response-body byte counts. For a large test resource, the actual body was not retained in the log. DoS-related checks used limited rates and concurrency, and transmission stopped once the anomalous behavior was confirmed. Those choices make the evidence safer and more interpretable, while also reminding us that a controlled measurement is not a stress test of the public Internet.
From attack knowledge to a defensible anomaly
The framework's knowledge problem has two parts. Short descriptions of attack mechanisms—what an inconsistency does and why it matters—are valuable on every generation step. Long protocol specifications are different: they contain precise field rules, frame restrictions, state transitions, and error definitions, but placing all of that text in every prompt would dilute attention and consume context. H3Act therefore uses a hybrid retrieval design.
The first part distills attack knowledge into principles and keeps those principles continuously available to the agents. A principle is more useful here than a catalogue of payloads because it describes the relationship to look for: two layers may disagree about a boundary, a resource identity, or the amount of work implied by a request. This persistent context gives the generator a direction for exploration without requiring it to rediscover the same security idea in every batch.
The second part retrieves precise protocol material when needed. RFC content is organized into logical sections and made available through a retrieval tool, so an agent can consult the relevant rule instead of relying on a vague memory of the standard. That division reflects a real engineering trade-off. Static context is good for compact, high-value concepts that should shape every test. Dynamic retrieval is better for long, detailed rules whose exact wording matters only for a particular case.
The Generator and Analyzer then take different roles. The Generator combines the available principles and protocol rules to produce concrete HTTP/3 requests. Its job rewards variation, but its output must still satisfy the format required by the sender and the protocol's validity conditions. A syntactically invalid request that the edge discards immediately provides little information about the translation path. H3Act uses a constrained request format so that the model spends more of its capacity choosing meaningful cases than repairing its own output.
The Analyzer receives the generated cases together with the observations collected at both ends. It compares the intended HTTP/3 semantics, the client-side result, the translated HTTP/1.1 request, and the origin response. It looks for a specific conversion anomaly and proposes a direction for the next batch. Its role favors careful, repeatable interpretation rather than maximum novelty. Separating the two roles also avoids forcing one conversation to be both highly divergent and highly deterministic at the same time.
That separation reduces two kinds of pressure. Long interaction histories can dilute an agent's attention, especially when old cases are no longer relevant to the immediate decision. And a single temperature or style of reasoning is poorly suited to both creative generation and meticulous comparison. H3Act keeps the Generator comparatively stateless, carrying forward the Analyzer's useful feedback rather than every prior exchange. It also periodically compresses older interaction history into a technical summary, retaining short-term detail for recent batches while preserving higher-value long-term lessons.
The operating loop is easier to understand as a set of handoffs than as a collection of model features.
The loop has a useful asymmetry. The Generator is allowed to explore a broad space of cases, but the Analyzer must narrow that space into a claim about a particular difference. The controlled origin provides the second observation needed to make that claim. Human experts then decide whether the reported difference is genuine, already known, or a false alarm. Removing any one of those stages would change what the result means: generation without comparison is speculation, comparison without a valid case is noise, and automated comparison without review is not a reliable security finding.
The study's evaluation metrics reflect this pipeline. In one runtime evaluation, the researchers varied the number of execution batches on Cloudflare and repeated each configuration. The framework generated an average of 74.4 cases over ten batches, 136.5 over twenty, and 215.6 over thirty. The corresponding total execution time increased as batches accumulated. These figures show how the system scales its own work with longer exploration; they are not throughput claims for every CDN or every model.
For detection accuracy, the paper ran five independent experiments for each of the nine tested services, with ten batches in each run. It collected 450 batch reports and used three human experts to review alerts. A successful discovery, in the study's definition, required a valid generated HTTP/3 request and an Analyzer report that identified a specific anomaly from the two ends of the path. Human review then checked whether the request, translated request, and logs actually supported the report.
The results were useful but not perfect. The services showed roughly three to four average unique anomalies in the reported batch-level results, while the false-positive rate ranged from 10.3% to 15.8% across the tested CDNs. The first reported anomaly generally appeared within the first few batches. Those numbers support the claim that knowledge-guided exploration can find meaningful behavior early enough to be practical for a study. They do not measure recall, severity, exploitability, or the probability that an untested vector is absent.
The false-positive definition matters. It is the ratio of manually confirmed false alarms to all anomalies reported by the model in an experiment. The authors did not determine whether each false alarm arose from hallucination, an ambiguous observation, or another cause. A rate in that range therefore does not tell us that the remaining alerts are equally severe or equally reproducible. It tells us that human verification was a necessary part of the measurement process.
The ablation results add another qualification. The study compared single-agent and dual-agent arrangements and tested the effect of including retrieval-augmented knowledge in selected configurations. In those experiments, the single-agent design did not provide a clear generation-time or output-token advantage, while its input cost was higher. The dual-agent design with retrieval generally produced lower false-positive rates and, in some comparisons, more detected anomalies than the alternatives. These are observations from the selected configurations and targets, not a universal proof that every two-agent system is better.
H3Act also pays a price for its flexibility. LLM-based generation is slower and more operationally expensive than rule-based binary fuzzing, and the number of packets produced per unit of time is lower. Safety alignment can occasionally cause a model to refuse a research-context request, while hallucination can cause the analyzer to report a discrepancy that the logs do not support. The current study uses one local base-model configuration and focuses on one translation direction. Broader protocol coverage, other reverse proxies, and stronger models remain future work rather than established results.
The most defensible interpretation is therefore neither “the model found everything” nor “the model's warnings are unreliable.” H3Act supplies a way to turn dispersed security knowledge into targeted black-box questions and to connect those questions to origin-side evidence. Its reported cases become stronger when the protocol rule, the conversion difference, and the consequence line up. They become weaker when the result depends on an unobserved internal assumption, a deployment detail outside the test, or a model judgment that human review cannot reproduce.
This is the standard that keeps the translation boundary in view. The point of measuring a CDN is not to count unusual requests or to treat a vendor name as a finding. It is to determine whether the edge preserved the request's meaning as it moved from HTTP/3 to HTTP/1.1, and then to ask which downstream component absorbed any difference. With that evidence chain in place, semantic drift can be separated from origin cost, cache impact, parser confusion, and operational disagreement instead of being collapsed into one alarming label.