Serving Inside the Training Loop
SGLang’s serving state becomes harder to reason about when the same GPUs alternate between rollout and training. This chapter follows that lifecycle: releasing memory without destroying the process, resuming it for the next rollout, and refitting new weights without letting old KV state masquerade as current state. The practical judgment is architectural: distinguish what the training system owns, what the serving runtime owns, and which transitions require idleness, synchronization, flushing, or versioning.
Reinforcement-learning systems repeatedly generate samples, evaluate or train on them, update model weights, and generate again. Rollout and training are both memory-intensive. If they share hardware, leaving all serving weights, KV cache, and CUDA-graph allocations resident can prevent training from fitting. Restarting the server for every training step, however, throws away process state and repeats startup work. SGLang’s RL integration treats the runtime as a library that can participate in this loop rather than as an isolated endpoint that never changes.
When serving and training share a GPU
The first lifecycle is memory occupation. During rollout, SGLang holds model weights, KV-cache state for active or retained requests, and any execution structures needed by the selected backend. When the training side needs the device, the engine can be asked to release selected regions of memory while keeping the server process alive. The guide describes fine-grained sleep and wake behavior for KV cache, weights, and CUDA-graph memory.
Keeping the process alive matters because a restart is not just a process-management event. It can require loading weights again, rebuilding runtime state, and recapturing CUDA graphs. SGLang’s memory-saver path is designed to preserve the serving process and restore the released occupation for the next rollout. The gain is not that the memory disappears at no cost; it is that the handoff can be made explicit and reversible instead of repeatedly reconstructing the entire serving engine.
The release transition has a guard: the engine must be fully idle. An active request may still be reading its KV pages, writing output, or holding a phase-specific disaggregation buffer. Freeing those resources underneath it would turn a memory optimization into a correctness failure. The release operation therefore asserts that no request is in flight before it pauses the selected memory regions.
The caller can release KV cache, weights, CUDA-graph allocations, or all supported regions. Releasing KV cache also flushes the serving cache, so the next rollout rebuilds the state it needs. This is the correct behavior when old request context must not survive the handoff, but it means the first requests after resume may pay cold-prefill costs. Releasing weights requires an additional condition when a CUDA-IPC weight cache is active: SGLang refuses the operation because a daemon and peer engines may still point at the shared device memory. Freeing it from one process would leave those peers referencing released storage.
Training then takes ownership of the available device memory. The serving process remains present but is not allowed to behave as if its previous allocations were still usable. On resume, SGLang restores the selected memory regions and synchronizes the model state across the relevant ranks.