Overview

FuzzGPU's evaluation on Vortex and OpenGPGPU uses Verilator RTL simulation, 20 core-hours per DUT, 10 seeds, and 100–10,000-instruction programs. It separates completion, generation throughput, effective throughput, coverage, ablation, and bug discovery rather than treating “performance” as one metric. Under the reported settings, FuzzGPU reaches higher productive execution than adapted Cascade and DiveFuzz baselines, while the ablations link depth to its stack, barrier planner, and taint. The evidence is bounded to two open-source designs and does not prove complete verification or universal superiority.

Learning outcomes

Completion measures executed depth, not correctness; generation throughput alone can reward tests that later abort.
On the reported completion experiment, FuzzGPU's median was 100% versus 5.0% for Ported-DiveFuzz and 45.4% for Ported-Cascade.
The results support target-aware productivity on two open-source GPUs, not coverage completeness, proprietary-GPU portability, or universal superiority.

Keywords

FuzzGPUVortexOpenGPGPUcompletioncoverageablation

What FuzzGPU's Results Actually Show

FuzzGPU's evaluation is designed to answer several different questions about its GPU-specific model, not one question called “performance.” This chapter separates what was compared on Vortex and OpenGPGPU, what completion, throughput, coverage, and ablation each measure, and how far the results can be generalized. That separation supports a more useful judgment: the paper provides evidence that its generator reaches more productive execution under the tested conditions, but no single percentage proves complete verification or universal superiority.

What was compared and why

The study evaluates FuzzGPU on two real-world open-source GPU designs, Vortex and OpenGPGPU, using RTL simulation with Verilator. The main fuzzing campaigns allocate 20 core-hours to each device under test and use 10 distinct initial seeds. Test programs vary in length from 100 to 10,000 instructions, and the instruction pool covers the supported integer, multiplication and division, atomic, floating-point, vector, and target-specific extensions.

Those details define the experiment's scope. The result is about the behavior of these target revisions under the stated simulator, resource budget, and instruction pool. It is not a measurement of every GPU architecture, every compiler-generated workload, or every possible fuzzing environment. The two targets are valuable because they are open-source designs for which RTL behavior can be inspected and instrumented, but they are not a representative sample of the entire commercial GPU market.

The comparison baselines require equal care. The paper says that no publicly available GPU hardware fuzzer was available for a direct comparison. The authors therefore adapted two state-of-the-art CPU fuzzers, Cascade and DiveFuzz, to GPU instruction sets. The ports replace the relevant instruction encoding and toolchain backends and add the initialization needed to launch GPU warps, while using the same differential framework for bug checking. They are useful controls for the question “what happens when a CPU-oriented generator is given a minimal GPU adaptation?” They are not native GPU fuzzers, and their results do not establish that every CPU-fuzzing technique fails on every GPU.

This distinction is especially important when reading a speedup. If FuzzGPU emits more instructions per second than a ported CPU generator, that may reflect the fact that the ported generator repeatedly constructs tests that abort or roll back. A raw count of emitted instructions can reward wasted work. The paper consequently measures several layers of productivity.

Instruction completion rate asks how much of a generated program actually executes. The paper measures the ratio of executed instruction program counters to total generated instructions using an ISS oracle. It is the closest metric to the question “did the input provide a deep experiment?” A high rate means that later control flow, barriers, memory operations, and functional units had a chance to run; it does not say that their results were correct.

Generation throughput asks how quickly the fuzzer constructs instruction streams. This matters because construction consumes campaign time, but it says nothing by itself about whether the streams survive execution.

Published by DeepMox

FAQ

What does instruction completion rate measure?

Instruction completion rate measures the share of generated instructions that actually execute, using executed program counters over total generated instructions. It indicates experimental depth, not result correctness.

Why are FuzzGPU's results not universal GPU benchmarks?

FuzzGPU's results are not universal GPU benchmarks because they come from Vortex and OpenGPGPU, fixed simulator conditions, stated budgets, seeds, and adapted CPU-fuzzer baselines. They do not predict every commercial GPU or workload.

What do FuzzGPU's ablations show?

FuzzGPU's ablations show that removing the generation stack, barrier planner, or taint propagation damages different parts of productive execution. In the reported OpenGPGPU settings, completion fell to 5.6%, 88.6%, and 17.1% respectively; these effects are not additive or universal.

Previous9 minFuzzGPU: GPU Differential Oracles

Next13 minFuzzGPU: Bug Diagnosis and Security Impact

FuzzGPU: Results on Vortex and OpenGPGPU

What FuzzGPU's Results Actually Show

What was compared and why

FAQ