Checkpoint: From Architecture to Usable Throughput

Nemotron 3 Super’s first architectural choices can now be connected into one practical question: what happens to a request as it moves through recurrent state, global attention, sparse experts, and speculative decoding? This checkpoint uses those already-established mechanisms to build a request-to-throughput model. The goal is not to repeat their definitions, but to identify which bottleneck each one addresses, what cost remains, and why the resulting efficiency is conditional rather than automatic.

Consider a long software-engineering request. The model may receive a repository description, issue details, prior observations, and tool results. It must preserve enough context to reason about the task, transform each new observation, choose specialized computation when useful, and emit a long sequence of actions and explanations. The request is successful only if the system remains accurate enough and responsive enough across that entire path. A fast isolated operation is not sufficient if another part of the path becomes the bottleneck.

One request crosses several boundaries

The first boundary is between the incoming context and the sequence state maintained by the backbone. Most of Nemotron 3 Super’s sequence processing uses Mamba-2 blocks, whose generation path maintains a constant-sized recurrent state. This limits the growth of one kind of memory burden as the model continues to generate. The state is not a verbatim archive of the repository or the conversation. It is a learned, evolving representation whose usefulness depends on what the model preserves as the sequence advances.

At selected points in the 88-layer stack, the representation crosses into an attention anchor. The anchor provides an opportunity for full-token interaction and long-range information routing. It keeps an attention key-value state, so it does not remove all context-related memory cost. Its role is narrower and more strategic: it restores direct global interaction at points where the model’s learned recurrent path may not be enough to connect distant information. The backbone therefore alternates between compact persistence and selected global access.

The next boundary is the expert route. Instead of applying every available expert to every token, the sparse layer selects a subset. LatentMoE first projects the representation into a smaller latent space, performs routing and expert computation there, aggregates the selected outputs, and projects back to the full model dimension. The token receives conditional capacity without sending a full-width representation through the entire selected-expert route. The route still requires a gate, expert movement, and full-width work outside the targeted bottleneck, but the expensive routed payload is smaller.

The last boundary in this path is decoding control. The main model produces a representation from which the shared MTP head can draft multiple future tokens. The main model then verifies the proposal. A prefix can be accepted, allowing the system to advance several output positions after one verification step; a rejected suffix does not become authoritative merely because the draft proposed it. This changes the number of sequential steps the system may need without changing which model controls the final decision.