CuSafe's Architecture: Tags, Bounds, and Compiler Checks
CuSafe answers the GPU metadata problem by distributing responsibility across an allocator, the pointer representation, the buffer itself, the compiler, and the runtime boundary. This chapter follows one allocation through that system. The goal is to make two distinctions clear: which facts travel with a pointer, and which facts remain attached to the memory object; and which part of the system creates, preserves, or checks each fact.
That division matters because CuSafe is not a single table consulted by a universal runtime. It is a coordinated path from allocation to instrumented dereference. Once that path is visible, the later spatial and temporal checks become easier to understand: one set of metadata describes where an access is allowed to go, while another helps establish whether the pointer still refers to the intended object.
From allocation to checked access
A normal CUDA program begins with source code that will eventually become GPU instructions. CuSafe inserts itself into that path through an LLVM transformation pass. The pass processes the program's intermediate representation and inserts validation instructions before pointer dereferences, as well as updates when metadata must change. NVIDIA's nvcc then compiles the instrumented representation into the executable. The prototype also supplies a dynamically preloaded library that intercepts memory-management APIs such as cudaMalloc, so allocations can receive the metadata that the inserted checks expect.
The allocator is the first active component. Instead of returning only the ordinary address of a payload, it creates a buffer layout with metadata and constructs a tagged pointer. The tag is not the exact byte count of the object. It carries compact information that helps identify the buffer's alignment class, distinguish pointer categories, and participate in later identity checks. The allocator and compiler must agree on the meaning of those fields. If the allocator produces a pointer with one interpretation while the instrumented code assumes another, the checker would fail before it could protect anything.
The compiler is the second component. It places the check at the operation that matters: a pointer dereference. A check can inspect the pointer's validity information, use its alignment-related information to locate the buffer's metadata, and compare the requested access with the exact bound stored there. The compiler also inserts the updates needed when a pointer or an allocation changes state. This is why the prototype is more than a library that watches API calls. An allocation hook knows that memory exists, but it does not automatically see every arithmetic operation and every later dereference performed inside a kernel.
The runtime boundary connects the two. The dynamically preloaded library handles the allocation interfaces that the compiler-inserted code cannot infer from ordinary program instructions alone. It can attach the in-band metadata at creation and clear or validate it at explicit deallocation. In effect, the allocator owns the initial object description, the compiler carries the checks into the kernel, and the runtime hook supplies lifecycle events that are otherwise outside the kernel's local instruction stream.
Consider what happens when one global buffer is created and later used. The allocation call enters the intercepted path, where CuSafe can reserve the buffer's metadata area and return a pointer carrying the compact tag. The application's CUDA code may then perform arithmetic on that pointer and pass it into a kernel. Because the LLVM pass has already inserted checks around the kernel's dereferences, the access does not rely on the application remembering to call a separate validation routine. The check uses the pointer representation to find the buffer-side information, tests the requested range, and only then permits the memory operation. When the allocation is released, the runtime event changes the buffer metadata so an old pointer cannot simply continue to treat the object as live.
This sequence also shows why the parts cannot be evaluated independently. A tagged pointer without a compiler check is only an encoded value. A compiler check without allocator metadata has nowhere reliable to retrieve the bound. An allocation hook without pointer instrumentation cannot see all later uses. CuSafe's contribution is the interface among these components: each one supplies state that another component needs at a particular transition in the allocation's life.
The control flow has an important failure path. If an access is invalid, CuSafe does not treat the metadata lookup as a best-effort hint and continue with the operation. The spatial checking process described in the paper rejects the access when either the pointer has been invalidated by arithmetic or the requested range lies outside the exact bound. This placement makes the check part of the memory access's precondition. It also explains why instrumentation reach matters: code that never receives the checks, or an allocation path that never receives the metadata, sits outside the same protection model.
The prototype's use of an LLVM 21 transformation pass and a dynamic library is therefore a deployment choice with a technical consequence. It avoids requiring a custom hardware unit or a proprietary compiler modification, but it assumes a compilation and allocation path that the prototype can intercept. The paper explicitly does not present this as support for every CUDA mode or JIT path. “Commodity deployable” means that the required capabilities exist on ordinary NVIDIA systems; it does not mean that no build integration or compatibility work is required.
Metadata lives in two places
The most important architectural choice is the hybrid split between pointer facts and buffer facts. A pointer carries compact information about its alignment class, its memory category, and its identity. The buffer stores the exact bound at the beginning of its allocation. The pointer helps the checker find the correct metadata anchor; the buffer supplies the precise size and a place whose state can be changed when the object is no longer live.
This is not a generic tuple of labels attached for convenience. Each field answers a different failure question. The alignment information helps locate the buffer and constrain pointer arithmetic. The type information tells the checker whether it is handling a global or a local representation. The identity information helps distinguish an old pointer from a current allocation. The exact bound answers the range question, while the buffer's current value answers whether the object still has a live size. Keeping those questions separate lets later checks fail for the right reason instead of treating every bad access as one undifferentiated tag mismatch.
Why not store the exact bound entirely in the pointer? A pointer is copied. If the only record of validity lives in one pointer value, invalidating the allocation may not invalidate copies that were made earlier. Conversely, why not keep all metadata in a separate table? The previous chapter established that separate lookups can create costly, non-broadcastable traffic for parallel accesses. CuSafe's split answers both problems: the pointer carries enough information for a fast route to the object, while the object remains the authority for exact bounds and liveness.
The alignment information has two related jobs. First, it describes a size class that lets CuSafe arrange or interpret the pointer's address so the lower offset bits can be cleared to reach the beginning of the buffer. Second, it gives the checker a way to notice when pointer arithmetic has changed information that should have remained stable. The aligned size is not treated as the final boundary. It is a compact index and integrity signal that leads to the exact size stored in-band.
That exact size matters because real allocations do not all fill their alignment region. Suppose an application requests a buffer whose size is not a power of two. A rounded-up tag can identify a region large enough to contain the allocation, but it cannot by itself distinguish the requested object from the padding created by alignment. CuSafe places the precise bound at the buffer's beginning so the final range check can use the size the program actually requested. This is the difference between finding the object's neighborhood and checking the object's boundary.
The bound is useful only if the pointer can be brought back to the correct metadata anchor. CuSafe uses the alignment information to clear the relevant offset bits, so a pointer to the middle of a buffer can still lead to the buffer's beginning. That operation is safe only while the protected portions of the pointer remain unchanged. If arithmetic crosses the region whose bits carry metadata, the pointer is marked invalid rather than being allowed to redirect the lookup to an unrelated object. Exact bounds and pointer-integrity checks therefore reinforce each other: one supplies precision, and the other protects the route to that precision.
The buffer-side location also supplies a liveness transition. When a global buffer is freed, the design clears its exact bound. For local buffers, invalidation occurs when the function exits because there is no corresponding explicit free call. A later dereference encounters a zero or otherwise invalid bound rather than a still-plausible size. This does not finish the temporal-safety problem—reuse can create metadata confusion—but it explains why the buffer, not just a pointer copy, must participate in the state change.
The GPU MMU carries identity without changing the physical object
For global memory, CuSafe uses the GPU's memory-management unit and page-table structure to arrange virtual-address bits that carry alignment information while preserving the mapping to the intended physical page. In simple terms, the virtual address is given a controlled pattern that the checker can interpret, but the data still reaches the allocation the program intended. This is a software use of existing address translation rather than a new hardware tag store.
The idea depends on which address bits the CUDA runtime leaves available. The paper describes fixed and controllable portions of the virtual address and uses the available bits for alignment and identity information under the cited NVIDIA constraints. The exact bit positions belong to that platform example; the general point is more useful than memorizing them. CuSafe turns part of the virtual-address layout into metadata while respecting the runtime's rules for valid addresses.
This is why the paper describes the MMU as an enabling interface. If the page tables determine how a virtual address maps to a physical page, then carefully arranged virtual addresses can carry information without allocating a separate metadata object for every pointer. The approach is still bounded by what the vendor exposes. A GPU with an MMU in the abstract is not automatically a supported CuSafe platform if its APIs do not permit the allocation and address control that the design needs.
The technique also separates virtual organization from physical consumption. CuSafe's alignment policy changes where a buffer appears in the virtual address space; the paper explains that it does not need to allocate physical memory merely to fill the padding between aligned addresses. That is important to the design's practical claim, because a virtual layout that improves metadata lookup would be less attractive if every gap consumed a corresponding amount of device memory. The separation does not make the layout free: address bits and runtime rules remain constraints, and the metadata field at each buffer still has a cost.
Local and shared memory require a different treatment. Their addresses are managed by the NVIDIA runtime in ways CuSafe cannot reshape through the same page-table configuration. The compiler therefore creates pseudo-pointers whose metadata bits are meaningful to instrumented code but must be removed before the actual dereference. A pseudo-pointer can travel through the instrumented part of the program carrying the information needed for validation; the raw pointer is restored at the boundary where the runtime expects an ordinary address.
The same stripping rule matters when a local or shared pointer crosses into an uninstrumented external function. That function does not know how to interpret CuSafe's metadata bits. Passing the tagged representation unchanged could make a valid pointer look malformed to the callee. CuSafe removes the tags before that call, accepting a compatibility boundary in exchange for keeping the instrumented path understandable. The paper notes that global pointers behave differently because their tagging is arranged through the address space and allocation interception, but that distinction should not be generalized beyond the described prototype.
The result is a system with several interfaces rather than one universal pointer format. Allocation crosses from the runtime API into the CuSafe allocator. Program representation crosses from ordinary LLVM intermediate code into an instrumented form. A tagged pointer crosses from metadata-aware compiler logic to a checked dereference, while local and shared pseudo-pointers must be untagged before crossing into code that was not instrumented. Each interface has an owner and a failure mode. Missing allocation metadata breaks the later lookup; an altered tag can make the pointer invalid; an unstripped pseudo-pointer can break an external call; an uninstrumented code path may not receive the same protection at all.
That complete path is the architectural model to keep in mind. The allocator creates a buffer and a tagged pointer. The compiler ensures that dereferences consult both. The runtime supplies allocation and lifetime events. The MMU helps global pointers carry controlled identity, while pseudo-pointers provide a compatible representation for local and shared memory. The pointer is a guide to the object's metadata, not the whole object record; the buffer is the authority for exact bounds and liveness, not a passive payload. The next analysis can therefore ask two separate questions: does the requested range fit, and does the pointer still identify a live object?