The Isolation Model: From GPU Sharing to Separate L2 Slices
This chapter examines the part of NVIDIA Multi-Instance GPU (MIG) that makes its security promise concrete: the path from a running CUDA kernel to the memory system, and the physical resources assigned to each instance along that path. The goal is to understand what the partition prevents, not to treat the word “isolated” as a complete explanation. Once the cache hierarchy and resource boundaries are clear, the difference between blocked cache eviction and a still-observable microarchitectural effect becomes much easier to judge.
A GPU is a pipeline of memory decisions
A GPU is not one processor with one cache. It contains many streaming multiprocessors, or SMs, that execute highly parallel workloads. A CUDA program submits kernels from the CPU, and the GPU schedules the kernels' thread blocks across available SMs. The threads execute in groups called warps, allowing many similar operations to proceed at once. This arrangement is why GPUs can process the same kind of operation over large arrays, images, graphs, or neural-network tensors efficiently.
Parallel arithmetic is only half of the story. A kernel also has to fetch instructions and data, and device memory is much slower to access than the small caches close to an SM. Each SM therefore has private L1 data and instruction caches. The GPU also has a larger L2 cache farther out in the hierarchy. The L2 is organized into slices and serves as a last-level cache for the SMs before requests reach device DRAM. An access that finds its data in L1 takes a different path from one that reaches L2, and both differ from an access that must wait for device memory.
That path gives a workload several possible sources of delay. A request can wait because its own data is not in the nearest cache. It can compete with other requests for an interconnect, an L2 slice, a memory controller, or a DRAM channel. It can also be affected by instructions whose purpose is not to fetch a particular data value but to impose an ordering condition on memory operations. These are different mechanisms. A useful isolation model must say which one it is addressing.
Without MIG, separate CUDA contexts normally time-share the GPU. A hardware scheduler gives each context a turn, so the contexts are isolated as software contexts but do not necessarily run at the same time. MIG changes the operating arrangement: contexts assigned to different instances can execute in parallel because the GPU has allocated portions of its hardware to those instances. That parallelism is the source of MIG's value for multi-tenant use, but it also means that the boundaries between the portions matter while workloads are active simultaneously.
The most important boundary in this discussion is the L2 cache. L1 caches belong to individual SMs, but L2 slices sit at a level where a large amount of traffic converges. A request may enter the L2 through a crossbar path, be served by a slice, and continue through a memory controller if the needed line is not present. MIG's promise is not merely that different users receive different SMs. It assigns each instance a distinct set of memory-system resources so that a workload's ordinary data traffic is less exposed to another workload's traffic.
That assignment changes the kind of interference a reader should expect. If a kernel runs on an SM belonging to one instance, its threads use that instance's compute capacity. When those threads request data, the request follows the instance's allocated path through the cache and memory system. Another instance can still be executing at the same time, but it is not supposed to consume the first instance's assigned SMs or L2 slices merely by issuing more work. This is the performance side of the boundary: each workload has a defined place in the machine rather than an informal claim on whatever capacity happens to be free.
The security side follows from the same arrangement but asks a different question. A tenant can learn from a neighbor only if some property of the neighbor changes something the tenant can observe. Shared cache capacity is one such property. If an instance owns neither the victim's cache sets nor the victim's data path, the ordinary replacement signal becomes unavailable. But the absence of that signal says nothing by itself about control logic that may coordinate memory operations across a broader scope. Architecture diagrams are therefore necessary, but they must be read as a map of resources, not as a proof that all effects stop at the same lines.
The diagram is deliberately a path rather than a component inventory. It shows where a normal data request goes and where MIG's physical assignment creates separation. It also shows the limit of what that picture can establish. If the line is stored in one instance's L2 partition, an ordinary request from another instance does not thereby gain access to it. But the diagram does not prove that every control operation touching the L2 system is local to one partition. That is the unresolved interface the research examines.
There is also a timing reason to keep the levels separate. An L1 hit, an L2 hit, and a device-memory access can all return the same logical value while taking different amounts of time. A measurement made by a workload in one instance therefore does not have to observe the neighbor's value to learn that something about the memory system changed. It only needs its own requests to experience a repeatable difference. The paper later exploits this kind of observation, but the architecture already tells us why timing is a distinct surface from data ownership.
The same distinction applies to the word “shared.” The L2 is shared by the SMs within the overall GPU architecture, but MIG assigns slices to instances for ordinary operation. A statement that an operation has an L2-level effect across instances does not mean that every slice becomes a common pool again. It means that the operation's handling may have a scope wider than the cache lines it stores. Keeping those meanings separate prevents a simple but inaccurate conclusion that the partition either works perfectly or does not exist.
What MIG actually assigns
NVIDIA's A100 user guide describes MIG as spatial partitioning that can create up to seven independent GPU instances, each with its own memory, cache, and compute streaming multiprocessors. The guide highlights two related outcomes: memory isolation among parallel workloads and physical allocation of resources. The exact number of instances is device-dependent; “up to seven” is a description for the A100 configuration, not a universal property of every MIG-capable GPU.
The paper gives a more detailed account of the resources involved. An instance receives its own SMs, a dedicated group of L2-cache slices, selected crossbar ports, memory controllers, and DRAM channels. These assignments divide both computation and the route by which data moves. They are intended to let workloads run concurrently with more predictable performance and less opportunity for one workload to consume another's assigned capacity.
This is a stronger arrangement than simply putting multiple jobs into separate software processes. A process boundary can control names, permissions, and virtual addresses while the hardware below remains shared. MIG adds a hardware allocation boundary. If a tenant receives a particular collection of SMs and L2 slices, another tenant in a different instance cannot normally make the first tenant's cache slice part of its own working set. If a tenant receives a portion of the memory controllers and DRAM channels, ordinary traffic is also less likely to contend for exactly the same physical path.
The word “normally” matters because the protection being described has a specific shape. It is strongest when the threat depends on shared capacity. Consider a conventional cache timing attack. The observer first places its own lines in selected cache sets. A victim then runs. If the victim uses the same sets, the observer's lines may be displaced. The observer checks its own access time or the presence of its own lines and infers that the victim was active. The technique does not require the observer to read the victim's data, but it does require some shared cache structure on which both workloads operate.
MIG's L2 partition removes the necessary overlap in the tested cross-instance case. The attacker and victim do not receive the same L2 cache sets, so filling one instance's partition does not provide a normal way to evict lines from the other's partition. The paper experimentally checked this point rather than assuming it from the architecture. The tested user-level LD, ST, and ATOM activities did not produce observable cross-instance L2 evictions. Most tested driver-level activities did not produce them either.
That result should be read as a real success of the resource boundary. It rules out an intuitive attack story in which a tenant simply floods its own cache accesses and watches the neighboring tenant's lines disappear. It does not show that all attacks are impossible, because a cache can expose more than line replacement. A request can be counted, ordered, delayed, flushed, or otherwise handled by logic that is not identical to the storage location of the line.
The distinction is important for both attackers and defenders. An eviction attack asks whether the victim can change the state of a cache set that the observer also controls. A timing attack can ask whether the victim's activity changes the service time of an operation the observer controls. Those questions may have different answers on the same hardware. The first can fail because the sets are assigned to different instances, while the second can remain possible if the operation being timed is affected by a wider memory-system event.
The experiments also show why an architectural claim should be checked empirically. It is tempting to infer complete isolation from a resource diagram: separate slices imply separate behavior. But the behavior of a complex GPU is determined by paths between components as well as by the components themselves. The paper profiles a simple detector precisely because a small, predictable workload makes an unexpected request or delay easier to see. That does not make the detector representative of a production application; it makes it a controlled instrument for testing the boundary.
The paper also reports one important lifecycle exception. Repeated CUDA context destruction caused an L2 flush across the entire GPU, evicting lines in all MIG partitions. This is not the same as ordinary cross-instance cache contention: it is a broad action associated with destroying a context, and modern applications typically do that near startup or shutdown rather than continuously. The finding matters because it shows that even a physical partition can have a lifecycle operation with wider scope, but it is not the frequent signal on which the main side-channel demonstrations depend.
The practical model is therefore layered. At the capacity layer, MIG assigns separate compute and memory resources. At the cache-line layer, the tested partitions prevent the standard eviction relationship between separate instances. At the operation layer, however, the question remains open: do all L2-related requests respect the same boundary, or can an ordering or coordination operation affect another partition without moving data into it? The paper's answer begins with memory barriers.
That distinction prevents two opposite mistakes. The first is to dismiss MIG because it does not make every microarchitectural effect disappear. The hardware partition still provides the resource isolation it was designed to provide. The second is to treat that resource isolation as proof of complete observational isolation. The cache-line result supports a narrower statement: separate instances do not share the tested eviction surface. It does not settle what another instance can learn from timing or from operations whose effects extend beyond one cache slice.
The architecture also explains why the later attack is subtle. A victim does not have to write into the attacker's L2 partition. The attacker can instead run an operation of its own whose completion time responds to an event elsewhere. If that response is repeatable, the observer can distinguish periods of activity from periods of idleness. The secret then appears not as a cache line but as a pattern in the time taken by the observer's own instructions.
That is why the next mechanism is not another tour of cache sets. It is an examination of how GPU memory ordering is implemented, which ordinary activities invoke it, and how an operation designed to preserve correctness can leave a measurable trace across a physical resource boundary.