How MoonBright Moves Page-Table Work to the GPU
MoonBright's architectural move is not to let arbitrary GPU code manage memory. It gives a trusted device-side path access to the page-table state needed for mapping, while the host continues to validate virtual ranges and obtain physical resources. The focus here is how that division becomes an operating system design: first a GPU-visible representation of the page-table hierarchy, then a small set of primitives that turn mapping into a parallel data operation. Understanding those boundaries lets the reader judge what MoonBright actually moves—and what it deliberately leaves on the host.
Making the page table visible without flattening its meaning
A GPU page table is not ordinary application data. It is a multi-level structure used by the hardware memory-management unit to translate a virtual address into a physical location. In a conventional system, the driver owns the authority to create and update that structure. A kernel can use the addresses that the table describes, but it is not normally allowed to treat the table itself as a writable array.
MoonBright begins by changing that visibility during CUDA context creation. The paper describes a bootstrap step that obtains the physical pages holding the hardware's page-table hierarchy and maps them into a specialized virtual range that GPU code can reach. The result is called the GPU Page Table, or gPGT. The hierarchy remains a hierarchy for hardware purposes, but MoonBright exposes a linearized view of its entries so a device kernel can address a particular translation entry with ordinary indexing.
The linearization is the key coordination idea. Given a virtual-page index, the host and device can calculate the entry's location from a common base address and the size of one page-table entry. The host does not have to ask the GPU where an entry landed, and the GPU does not have to walk a host-side data structure to find the destination. Both sides can derive the same location arithmetically. That avoids repeated round trips across the host-device interconnect while preserving a deterministic view of the translation state.
The bootstrap is a view, not a second translation system with an independent meaning. The physical pages that support the hardware hierarchy are made reachable through a device virtual range, and the linearized address is a convenient way to reach entries in that existing structure. A kernel can therefore perform the repetitive write where the GPU's memory system is fast, while the hardware still interprets the resulting hierarchy according to its normal translation rules. The difficult part is not inventing a new page-table format; it is making the existing state safely addressable and keeping the host's model of it aligned with device execution.
This does not mean that every virtual page has a complete set of lower-level structures waiting for it. Modern page tables are sparse: a range may require new directory levels only when a mapping reaches a region that has not been populated before. MoonBright therefore separates the act of creating missing hierarchy from the act of filling leaf entries. The first is a structural operation that must ensure the page-table tree exists. The second is the large, repetitive transformation that can be distributed across GPU threads.
The separation also clarifies who owns which facts. The host-side runtime still knows whether an address range is reserved, free, or valid for the current request. It still obtains the physical page-frame numbers, or PFNs, from the system's physical resource manager. The device-visible view supplies a fast destination for translation entries; it does not become an invitation for an untrusted kernel to rewrite arbitrary mappings. In MoonBright's stated deployment model, the mapping kernels are generated and launched by the trusted runtime.
That ownership boundary is essential to the “software-only” claim. MoonBright reports that it runs on commodity NVIDIA and AMD GPUs without modifying the GPU hardware. It does not mean that an unmodified vendor driver already exposes the required page-table view. The prototype needs driver and runtime changes to bootstrap that view, expose the low-level primitives, and enforce the intended coordination. Hardware compatibility and software integration are separate questions.
The diagram's important relationship is not the number of components. It is the direction of authority. The runtime coordinates the request; host metadata and the physical allocator supply decisions and resources; the PFN buffer carries trusted input to device execution; and device kernels write the page-table representation. TLB coherence remains a separate control path because a valid page-table write and a valid cached translation are not automatically the same thing.
From five primitives to a mapping pipeline
MoonBright organizes this division through five composable primitives. They are easier to understand as stages in a request than as a list of API names, because each stage owns a different state transition.
The first stage, alloc_va, handles the logical address space. It reserves a virtual range and updates the host-resident layout metadata that tells the runtime which portions are available or occupied. At this point, no physical memory has necessarily been committed, and no usable translation has been installed. The range is a promise about where a mapping may be placed.
The second stage, alloc_pfn, obtains the physical backing. It asks the low-level resource manager for page-frame numbers and records the allocation layout needed by later stages. The PFNs are more than opaque handles in this design: they become the input data from which device-side page-table entries can be formed. Physical allocation remains a resource-management operation, so MoonBright does not claim that GPU parallelism replaces the system allocator or removes capacity limits.
The third stage, pgt_create, handles missing structure. If the target virtual range reaches a part of the sparse page-table hierarchy whose directory levels do not yet exist, the runtime validates the request against the host's virtual-address state, obtains the backing needed for those directory pages, and installs the structural links. This is different from populating the final entries for the user's data pages. Without the hierarchy, there is nowhere for those leaf entries to live; without valid host metadata, a device kernel would not know whether it was allowed to create it.
This gives the pipeline two different correctness questions. Structural correctness asks whether the page-table levels and links exist in the right places. Mapping correctness asks whether each leaf entry points to the intended physical frame with the intended access bits. A request can satisfy one and fail the other: a hierarchy may exist but still contain no valid leaf mapping, or a batch of leaf entries may be prepared for a range whose structural path was never created. Keeping the stages distinct makes failures easier to reason about and prevents the parallel kernel from being responsible for decisions it cannot safely infer from page data alone.
The fourth stage, pte_pop, is MoonBright's central data-plane operation. The runtime first places the raw PFNs in a GPU-resident buffer. A generated GPU kernel then processes the mapping directives in parallel. For each target page, it combines the physical address with the requested permission bits, produces the hardware-compliant Page Table Entry, and writes that entry into the appropriate slot in the gPGT view. The source PFN data and the destination page-table entries both reside in device memory, so the operation is primarily high-bandwidth device-side data movement rather than a sequence of host-generated records sent one at a time.
The input and output sides matter equally. The PFN buffer tells the kernel which physical pages the host has acquired, while the linearized page-table view tells it where the corresponding translation entries belong. The kernel is not searching for memory, selecting a placement policy, or deciding whether a caller has permission to change an address range. Those decisions have already crossed the host boundary. Its job is to apply the mapping description consistently at a scale where independent work can run in parallel. This is the narrow operation that lets MoonBright use GPU throughput without turning page-table management into arbitrary device-side authority.
The distinction between pgt_create and pte_pop explains why the design can scale without pretending that all work is identical. Creating a missing upper-level structure involves discovering a structural gap and allocating the page that fills it. Populating many leaf entries involves applying the same transformation repeatedly over independent pages. The first needs coordination around the shape of the tree; the second is the natural target for thousands of parallel GPU threads.
Because pte_pop is expressed as a normal GPU kernel, it can participate in stream ordering. A runtime can enqueue the population operation before a dependent compute kernel in the same stream, so the consumer does not run until the mapping work has completed. Independent work in another stream can potentially proceed while the entries are being materialized. This is a different kind of ordering from a host call that blocks until a global device state is settled: the dependency is attached to the mapping operation and its consumers rather than imposed on every stream.
That overlap has a precise limit. The dependent kernel cannot read a mapping before its entries are ready, and a structural update or an operation that changes an existing address may still need host coordination. What can be removed is the requirement that every page-sized update be completed as a host-side control transaction before any related GPU work is queued. MoonBright changes the shape of the wait: a consumer waits for its producer mapping operation, while unrelated work need not inherit a device-wide barrier merely because a mapping was created.
The fifth primitive, tlb_flush, is deliberately not folded into every mapping. It invokes a host-initiated hardware shootdown when an update could leave a stale translation for the same virtual address—for example, an existing mapping is overwritten, permissions change, or an old address is recycled. A fresh-address mapping may avoid this step under MoonBright's deferred-coherence policy, but the primitive remains available for cases that do not satisfy the freshness condition. Page-table materialization and TLB invalidation therefore form two related but separate operations.
A complete mapping request can now be read as a controlled pipeline. The host reserves a range, acquires physical pages, and ensures the page-table structure exists. It then stages the PFNs and launches parallel PTE population on the GPU. The mapping becomes usable after the relevant stream dependency completes, subject to the coherence rule for that address. If the request changes an old relationship rather than creating a fresh one, the runtime adds the exceptional flush. Each transition has a clear trigger and owner instead of being hidden inside an undifferentiated allocation call.
For a newly requested region, this pipeline separates a logical promise from a usable pointer in a way that is easy to observe. First, the runtime records where the region may live. Next, the physical allocator returns the pages that will back it. If the table is sparse at that location, structural pages are installed. Only then does pte_pop write the leaf relationships that make the data pages reachable. The application can enqueue a consumer behind that operation, but it cannot skip the dependency simply because the address was reserved earlier. The design is faster where it replaces repeated host construction, not where it ignores the ordering needed to make a mapping real.
This pipeline also explains why MoonBright's improvement is not simply “use more GPU threads.” The threads are useful only after the host has provided valid input and a legal destination. A page-table entry encodes privileged translation state, and a physical frame must remain owned and resident for the mapping's lifetime. MoonBright moves the repetitive construction step across the boundary; it does not remove the boundary's security and resource obligations.
The architecture's promise is consequently precise. On commodity hardware, a software-modified runtime can make GPU-resident translation state reachable and use the device's bandwidth to populate many entries. The host still owns high-level address policy and physical allocation, and exceptional updates still need coherence handling. The remaining question is how MoonBright decides when an update is genuinely exceptional. A page table can be populated quickly, but a stale TLB can still remember an old meaning for an address. The next mechanism addresses that second half of the problem by controlling when virtual addresses are reused.