Teaching the Model to Operate

Nemotron 3 Super becomes an agent through a post-training lifecycle, not through a single instruction-tuning step. This chapter focuses on how the model moves from supervised examples to verifiable interaction: how the training objective controls reasoning length, how tools and repositories provide evidence about action quality, and how asynchronous reinforcement learning scales long-horizon work while introducing policy-lag and infrastructure risks. The result is a way to distinguish a model that can describe an action from one that has been trained to take, check, and revise actions in an environment.

The distinction matters because an agent’s output is not judged only by its final paragraph. A software-engineering agent must modify the right repository state. A tool-using agent must call the right interface with the right arguments. A search agent must gather useful observations rather than produce a plausible story. Those outcomes depend on the trajectory between request and answer, which is why Nemotron’s post-training pipeline combines supervised traces, executable environments, verifiers, and preference signals.

The post-training lifecycle changes what counts as a good response

The first stage is supervised fine-tuning. The report identifies a failure in a single-stage recipe: when the loss is averaged over output tokens globally, long responses contribute more tokens and can dominate the update. The model may learn strong reasoning behavior, but it can also become worse at tasks that provide a long input and require a short answer. In a long-context setting, that is a serious mismatch. The input may be large because the model must inspect evidence, not because the desired response should be long.

Nemotron addresses the conflict with two SFT stages. The first uses token-level averaging across the packed batch. This encourages the model to learn detailed reasoning and gives every output token its ordinary likelihood contribution. The second normalizes each conversation by its own output length before averaging across conversations. A long response still contributes its internal tokens, but it no longer receives disproportionate influence merely because it has more of them. Each conversation gets a more equal influence on the update.

This is a change in what the training system considers a representative example. The first stage treats output tokens as the basic unit of supervision. The second treats conversations as the basic unit. Neither objective is universally correct. Token-level averaging can be useful when detailed reasoning is the target; conversation-level averaging can protect shorter responses and long-input/short-output behavior. The two-stage sequence uses them for different purposes rather than asking one objective to solve both problems at once.

The sequence length changes with the objective. The first stage uses 256K packing, while the second uses 512K packing and includes samples up to 512K tokens. The shared MTP head remains active during SFT with an auxiliary loss scaled by 0.3, preserving the connection between representation learning and later speculative decoding. The model also trains reasoning-off, regular, and low-effort modes. It is not being shaped into a system that treats longer reasoning as the answer to every request.

These modes turn effort into a controllable behavior. Reasoning-off can serve situations where the user needs a concise answer or where hidden deliberation would add cost. Regular reasoning can support harder tasks. Low-effort reasoning can trade some deliberation for speed when the task does not justify a large token budget.