Overview

MoonBright’s Always-Fresh policy avoids many common-path TLB shootdowns by assigning new mappings virtual addresses that have not held a valid device-visible translation during the current coherence epoch. The first access may incur a TLB miss and page walk, but it does not need to invalidate an older same-address mapping. Freed ranges enter quarantine; at the ring buffer’s wrap boundary, MoonBright waits for relevant GPU work, batches invalidation, and begins a new epoch. This trades frequent synchronization for virtual-address pressure. The paper’s NVIDIA measurements also require platform-specific spacing for a prefetch window, while remapping, permission changes, and reclamation still need explicit coherence and lifetime ordering. The reported budgets and latencies are platform- and workload-specific, and the trusted page-table view raises security concerns in untrusted sharing.

Learning outcomes

A fresh virtual address avoids invalidating an older valid same-address translation, but its first access can still incur a TLB miss and page walk.
MoonBright amortizes reuse costs by quarantining old ranges and batching TLB invalidation at epoch boundaries, consuming virtual-address space meanwhile.
Always-Fresh does not cover remapping, permission changes, or reclamation; those operations still require ordering and explicit coherence handling.

Keywords

MoonBrightAlways-FreshTLBcoherencequarantineprefetching

How MoonBright Avoids Most TLB Shootdowns

MoonBright's second mechanism addresses the coherence problem left after page-table work moves onto the GPU: when is a newly written mapping safe to use without a global TLB flush? The answer is not that GPU translation caches suddenly become coherent. It is that MoonBright controls virtual-address reuse so a fresh mapping does not compete with an older valid translation at the same address. Following that lifecycle shows why the fast path can replace a large synchronization event with a first-touch miss, and where the flush remains unavoidable.

A fresh address changes the coherence question

A page-table update changes the system's underlying translation record. A TLB holds a cached record that the GPU may consult without walking the page table again. If an existing virtual address used to point to physical page A and is now made to point to physical page B, the page table and the cache can disagree. The GPU could continue using the old answer unless the old entry is invalidated. The same hazard appears when a mapping is removed or its permissions change.

MoonBright's Always-Fresh policy avoids that particular conflict on the common allocation path. When the runtime installs a new mapping, it chooses a virtual-address range that has not previously been used for a valid translation during the current coherence epoch. There is no old valid same-address binding for the device to reuse because the address has never represented a live mapping that the device could have cached. The page-table write still has to complete before a dependent kernel uses the range, but installing the first mapping does not require invalidating an earlier mapping at that address.

The first access is not free. With no warm translation, the GPU may miss in its TLB and walk the page table to find the newly installed entry. That is a local translation event tied to the accessing work, rather than a command that stops the device to invalidate translations everywhere. The distinction is important: a miss asks the hardware to discover the current entry, while a shootdown is a system-wide action used to remove potentially stale entries.

The paper's NVIDIA A100 microbenchmark makes the trade-off concrete. It reports about 533 cycles for a pre-warmed TLB hit and about 1,489 cycles for a forced miss that walks to a valid PTE—an additional 956 cycles for the miss in that test. By contrast, the conservative CUDA Driver VMM flush path followed by a load takes about 26.8 microseconds and is dominated by host-device synchronization. The comparison does not mean every first access costs exactly 1,489 cycles or every flush costs exactly 26.8 microseconds. It shows the different scale and location of the costs measured on that platform: a small device-side first-touch expense versus a host-mediated global coherence path.

The protocol also accounts for speculative translation behavior. A page walker or prefetcher may probe a nearby address and cache the fact that no mapping was present. MoonBright's reasoning treats such a negative result as different from a valid translation to an old physical frame.

Published by DeepMox

FAQ

What makes a virtual address fresh in MoonBright?

A virtual address is fresh when it has not previously been used for a valid translation during the current coherence epoch; reusing a freed pointer value may therefore be unsafe.

What happens when MoonBright’s fresh-address space reaches its wrap boundary?

At the wrap boundary, MoonBright waits for GPU work that may reference old ranges, performs a batched TLB invalidation, and starts a new epoch for recycling.

How does NVIDIA prefetch behavior affect MoonBright’s address policy?

On the tested NVIDIA hardware, MoonBright spaces fresh addresses around a 32 MB prefetch window; the paper’s AMD backend did not observe the same behavior or use that padding.

Previous8 minMoonBright Device-Side Page Tables

Next11 minMoonBright Virtual Stitching for Dynamic Workloads

MoonBright Always-Fresh TLB Strategy

How MoonBright Avoids Most TLB Shootdowns

A fresh address changes the coherence question

FAQ