Overview

Qwen3 transfers more than answer patterns to smaller variants through two distillation phases. Off-policy teacher sequences establish `/think` and `/no_think` behavior; on-policy distillation has the student generate its own states while teacher logits guide next-token choices through KL divergence. In the reported Qwen3-8B comparison, on-policy distillation outperforms the tested RL continuation on several math and coding measures while using 1,800 versus 17,920 GPU hours. Those figures describe one training comparison, not inference cost, memory, or guaranteed first-attempt reliability; MoE active parameters likewise do not settle deployment cost.

Learning outcomes

Off-policy distillation gives smaller Qwen3 models teacher-generated examples for reasoning patterns and mode switching.
On-policy distillation corrects student-generated states with teacher logits, addressing routes the student actually takes.
On Qwen3-8B, the tested on-policy route reached stronger reported scores using 1,800 rather than 17,920 GPU hours, but this is training cost.

Keywords

Qwen3DistillationReasoningTeacherStudentLogits

How Qwen3 Makes Reasoning Portable to Smaller Models

Qwen3's post-training creates a difficult transfer problem: a smaller model must inherit more than a collection of good answers. It must learn when to deliberate, how to respond directly, and how to use a limited computation budget without the flagship's full capacity. This chapter explains how the report's off-policy and on-policy distillation address that problem, then uses the Qwen3-8B comparison and the sparse Qwen3-30B-A3B design to separate training efficiency from the very different question of deployment cost.

From teacher examples to student-generated behavior

The distillation pipeline begins with a strong teacher and a lightweight student. The report applies it to five small dense models—Qwen3-0.6B, 1.7B, 4B, 8B, and 14B—and to the Qwen3-30B-A3B MoE model. The objective is not simply to shrink the parameter count. The student should acquire the mode-switching behavior that makes Qwen3 a family rather than a collection of unrelated small assistants.

The first phase is off-policy distillation. The team combines teacher outputs generated under /think and /no_think and uses those outputs to teach the student basic reasoning patterns and the ability to switch between modes. “Off-policy” means the sequences were generated by the teacher's policy, not by the student's current behavior. The student receives examples of what a strong model does, including the contrast between a deliberate path and a direct response, and learns from those examples as a starting point.

This is useful because a small student may not yet know how to explore a difficult problem. If training asked it to generate all of its own reasoning examples from the beginning, many of those examples could be short, incorrect, or repetitive. Teacher-generated sequences provide a stronger initial signal. But imitation alone has a limitation: the student will eventually encounter prompts and partial trajectories that differ from the teacher's examples. A student that only learns the teacher's completed paths may not know how to recover when its own generation takes a different route.

The second phase is on-policy distillation. Prompts are sampled, and the student generates its own response in either thinking or non-thinking mode. The teacher then supplies token-level logits for the same situation, and the student is fine-tuned by minimizing the Kullback–Leibler divergence between the student and teacher distributions. In plain language, the teacher guides the student's next-token preferences while the student is operating on its own sequence. The student is not merely copying a fixed answer; it is being corrected in the states its own behavior actually reaches.

That distinction explains why on-policy distillation can transfer both capability and control. The /think and /nothink examples establish the intended branches. Student-generated sequences test whether the student can enter and continue those branches. Teacher logits provide a richer signal than a final-answer label: they indicate how the teacher ranks possible continuations at each point in the sequence.

Published by DeepMox

FAQ

What is the difference between off-policy and on-policy distillation in Qwen3?

Off-policy distillation teaches Qwen3 students from teacher-generated sequences, while on-policy distillation has the student generate sequences and then uses teacher logits to guide those states. The latter addresses behavior the student actually produces.

Why does Qwen3 use on-policy distillation?

Qwen3 uses on-policy distillation because a student may drift from teacher examples; having it generate its own states lets teacher logits correct the routes it actually takes. This targets a limitation of fixed answer imitation.

Do Qwen3's distillation results show cheaper inference?

No. The report's 1,800 versus 17,920 GPU-hour figures measure the tested training routes, not inference or serving cost. They do not establish ten-times lower latency, memory use, or operating expense.

Previous10 minQwen3 Post-Training: Reasoning and Versatility

Next11 minQwen3 Benchmarks: What the Scores Show

Qwen3 Distillation: Reasoning at Smaller Sizes

How Qwen3 Makes Reasoning Portable to Smaller Models

From teacher examples to student-generated behavior

FAQ