Why GPU Memory Safety Needs a Different Sanitizer Design
CuSafe is a software sanitizer designed to detect memory corruption on commodity NVIDIA GPUs. This opening chapter focuses on the problem that gives CuSafe its shape: GPU applications increasingly run important machine-learning and scientific workloads, yet their CUDA programming model still relies heavily on memory-unsafe C and C++. The practical question is not simply whether a checker can identify a bad address. It is whether the checker can do so across thousands of parallel threads without turning the workload unusably slow, and whether it can be deployed on the hardware and toolchain developers already have.
That question places CuSafe in a specific relationship with NVIDIA's Compute Sanitizer and with earlier GPU safety proposals. Compute Sanitizer already supplies valuable checks; other systems obtain strong protection through custom hardware or narrower software techniques. CuSafe's significance is its attempt to join accurate spatial and temporal detection with ordinary NVIDIA hardware, an available LLVM toolchain, and a metadata design that fits GPU memory behavior. Understanding that combination—not memorizing a list of features—is the key to judging what CuSafe contributes.
CuSafe begins with a problem the GPU has inherited
GPUs were first associated with graphics, but they now execute general-purpose programs in machine learning, scientific computing, and computer vision. Those programs often manipulate large arrays and complex data structures through CUDA, whose low-level programming model inherits the hazards of C and C++. An incorrect index, an integer overflow in pointer arithmetic, or a stale pointer can therefore corrupt memory just as it can in a CPU program. The difference is that the corrupted data may sit inside a highly parallel computation whose result is expensive to reproduce and difficult to inspect.
The consequences are not limited to a crashed kernel. The CuSafe paper connects GPU memory bugs with failures reported in major machine-learning frameworks and with prior demonstrations of attacks that can alter model behavior or use corruption as part of a larger exploitation chain. A sanitizer is consequently useful at two levels. During development, it can expose a defect before it becomes a production reliability problem. From a security perspective, it can prevent a corrupted pointer from reaching the memory access that gives the bug leverage. It is not a complete defense against every attack, but it changes the point at which many failures can be observed and stopped.
The two broad classes of failure already suggest why the problem is more than a range check. A spatial error reaches outside the object that a pointer is supposed to address. A temporal error uses a pointer after the object has been freed or after a local variable's lifetime has ended. The address in a temporal failure may still look numerically valid; it may even refer to a new object. A useful sanitizer must therefore reason about both where an access lands and whether the object associated with that address is still the intended live object.
That distinction matters especially in GPU programs because an invalid value can be consumed by many threads before the symptom becomes visible. A bad write may change a tensor used later by a model, while a stale pointer may produce a plausible result rather than an immediate crash. The paper does not claim that CuSafe eliminates every such consequence, and it is not a complete security architecture. Its narrower contribution is to move detection into the instrumented access path, where spatial and temporal conditions can be checked before the GPU continues with the operation.
CuSafe is presented as a response to a gap in the existing design space. Hardware-assisted proposals can check memory efficiently, but they require hardware that ordinary developers do not possess. Some software tools work with existing devices but recognize only particular overflow patterns. Others depend on proprietary compiler or toolchain changes that are difficult for outside users to obtain. The challenge is therefore a three-way tension: broad detection, acceptable cost, and practical deployment rarely arrive together automatically.
The phrase “different sanitizer design” should therefore be read narrowly and concretely. It does not mean that GPU programs need a completely new definition of memory safety. They still need bounds, validity, and lifetime checks. It means that the location of metadata, the way addresses are formed, and the point at which checks are inserted must take the GPU's execution substrate seriously. A CPU design that is correct in principle may become too expensive when every thread performs a separate metadata lookup.
The hidden cost of checking memory in parallel
A useful starting point is the GPU memory hierarchy. Shared memory is a small, fast region available to threads on the same streaming multiprocessor. Local memory is associated with an individual thread and functions as stack-like storage. Global memory is persistent device memory that can be allocated and freed through runtime APIs and shared across GPU functions. These regions do not have the same ownership or lifetime, and the programmer does not have the same control over their addresses. A sanitizer that treats them as one uniform pool loses information that matters to both correctness and deployment.
Those lifetimes also change the kind of evidence a checker needs. A global allocation normally has an explicit allocation and free operation. A local buffer is tied to a function's execution and disappears when the relevant threads leave that function. A mechanism that can clear metadata at a cudaFree call still needs another way to recognize that a local stack frame has ended. This is one reason CuSafe later gives local and global pointers different identity information. Even at the opening level, the lesson is clear: memory safety follows the resource lifecycle, not just the numerical address.
Parallelism adds another constraint. Imagine a kernel in which thousands of threads process different elements of one array. A conventional out-of-band design can store bounds in a separate shadow table, but the threads may then query different shadow addresses because each thread has a different data offset. The memory system cannot necessarily combine those requests. The safety check has created another stream of traffic that competes for bandwidth and cache capacity with the computation it is supposed to protect.
CuSafe's paper uses this contrast to motivate in-band metadata. If the exact bound is placed at a common anchor associated with the buffer, threads working on different offsets can retrieve the same metadata location. The GPU may broadcast that result, so the metadata lookup can behave more like one additional transaction than one additional transaction per thread. This is an architectural rationale, not a universal proof that every in-band scheme is faster than every shadow scheme. Its importance is that metadata placement becomes part of the performance and security design rather than an implementation detail added afterward.
The same tension appears in earlier protection strategies. A canary placed around a buffer can detect an overwrite that reaches the guard value, but an overflow that skips past it may leave the canary untouched. A power-of-two allocation tag can make bounds compact, but it may treat unused alignment padding as valid space. Custom hardware can address these weaknesses efficiently, yet it changes the deployment assumption from “instrument the application” to “obtain a modified device.” CuSafe is trying to keep the stronger distinctions while using controls already exposed by commodity NVIDIA systems.
Compute Sanitizer is useful—but it exposes the gap
NVIDIA's Compute Sanitizer is not a straw man. The official documentation describes it as a functional-correctness suite distributed with the CUDA toolkit. Its memcheck tool detects and attributes out-of-bounds and misaligned accesses to global, local, and shared memory, can report hardware exceptions, and can check for leaks. Racecheck focuses on shared-memory data hazards. Initcheck reports uninitialized accesses to device global memory, and synccheck reports invalid uses of synchronization primitives. The suite can be placed around an application rather than requiring developers to redesign every allocation by hand.
That breadth makes Compute Sanitizer an important baseline. It shows that commodity GPU checking is possible and gives developers several ways to investigate a failing program. The documentation also acknowledges a subtle coverage issue: global allocations can be adjacent in virtual address space, so an overflow from one allocation may land in another and look ordinary. Optional padding can make such an access visible by creating a protected gap. The remedy illustrates the broader design problem: improving detection can consume memory or alter execution behavior.
Compute Sanitizer also demonstrates that “a sanitizer” is not one indivisible capability. Memory access errors, shared-memory races, uninitialized data, synchronization hazards, and leaks have different evidence and different reporting paths. A broad debugging suite can be valuable even when its most precise checks are too expensive for every workload or every stage of development. CuSafe addresses a narrower but central question: can the ordinary memory-access path carry enough identity and bounds information that strong spatial and temporal checks become practical for production-scale GPU programs?
CuSafe's argument is not that Compute Sanitizer detects nothing. In the paper's comparison, the authors report that Compute Sanitizer catches some spatial and temporal cases but carries much higher runtime cost on their tested workloads. Those figures belong to that experiment and hardware/software setup, not to every possible use of NVIDIA's tool. The more durable distinction is architectural. Compute Sanitizer's documented suite provides several checking modes; CuSafe reorganizes pointer identity, buffer bounds, and compiler-inserted checks to pursue lower-cost checking while retaining coverage for cases that simpler tags or canaries can miss.
That is the question this series will keep returning to: how can a sanitizer improve what it recognizes without making the GPU computation impractical or narrowing the paths it can actually observe? CuSafe's answer begins with a split between metadata held by the pointer and metadata held by the buffer, then extends through the GPU address-translation system and the compiler. The next step is to examine the constraints themselves—especially why a metadata lookup that looks small in a sequential program can become a systems problem when thousands of GPU threads perform it together.