Overview
MoonBright addresses a measured mismatch in GPU systems: kernels can be highly parallel while flexible memory allocation remains dominated by host-side page-table construction, propagation, and coherence work. The series follows the path from CUDA and HIP virtual-memory primitives to MoonBright’s split design: the host validates ranges and acquires physical pages, while trusted GPU kernels populate page-table entries; fresh virtual addresses defer most same-address TLB flushes to epoch-based reuse. That foundation can support virtual stitching, dynamic allocation, aliases, and selected training and inference gains without copying live data. The benefits remain conditional: physical capacity, granularity, reclamation, virtual-address space, platform behavior, and privileged-state security are not removed.
Learning outcomes
- On the tested A100 `cudaMalloc` path, page-table construction and propagation account for most reported latency as mapped ranges grow.
- MoonBright keeps validation, physical allocation, and structural page-table work on the host while trusted GPU kernels populate leaf entries in parallel.
- Always-Fresh virtual addresses replace many common-path same-address flushes with first-touch misses, then batch reuse-related coherence at epoch boundaries.
- Virtual stitching and aliases can improve layout efficiency for selected dynamic workloads, but physical limits, hardware rules, virtual-address pressure, and security remain.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
Why is GPU memory allocation slow in MoonBright’s target path?
In the tested A100 path, page-table construction and propagation dominate the reported `cudaMalloc` latency as mapped ranges grow, while physical allocation is a smaller component.
What does MoonBright move from the host to the GPU?
MoonBright moves bulk leaf page-table-entry population to trusted GPU kernels; the host still validates ranges, obtains physical pages, and creates missing structure.
How does MoonBright avoid most GPU TLB shootdowns?
It assigns fresh virtual addresses to new mappings, avoiding stale same-address translations on the common path; reuse and remapping still require epoch or explicit flushing.
Can MoonBright reduce GPU memory fragmentation?
It can reduce external fragmentation by stitching scattered physical pages into a contiguous virtual range, but it does not remove page-granularity waste or physical capacity limits.