Overview

For Mixtral 8x7B and Hunyuan-Large, active parameters are a selected-path measure, not a standalone efficiency score. Mixtral reports 47B total and about 13B active parameters; Hunyuan-Large reports 389B total and 52B activated under a different shared-plus-specialized route. Interpreting those counts requires weight memory, routing traffic, placement, batch shape, training history, and evaluation protocol. The reports support within-report observations but no common serving experiment or causal attribution from active count to capability.

Learning outcomes

Mixtral's 13B and Hunyuan-Large's 52B counts come from different route structures and are not identical denominators.
A lower active count does not imply a proportionally smaller system because weights, communication, placement, and workload shape remain.
Capability results reflect complete training and evaluation pipelines, not routing or active parameters in isolation.

Keywords

Mixtral 8x7BHunyuan-LargeParametersComputeMemoryPlacement

Mixtral and Hunyuan-Large: What Active Parameters Do—and Do Not—Tell Us

For Mixtral 8x7B and Hunyuan-Large, the active-parameter headline is useful only when it is read alongside the total parameter pool and the route that produced it. This chapter connects the two routing designs to the practical question readers will eventually face: does a smaller active path mean a smaller system? The answer is narrower. Active parameters describe a selected computation path; memory, routing traffic, placement, and workload shape determine how that path behaves in a real system.

Read the two counts as a pair

Mixtral 8x7B is reported with 47 billion total parameters and about 13 billion active parameters per token. Hunyuan-Large is reported with 389 billion total parameters and 52 billion activated parameters. It is tempting to put the two active numbers on a scale and call Mixtral more efficient, or to put the two total numbers on a scale and call Hunyuan-Large more capable. Neither move preserves enough of the architecture to be reliable.

The total count answers a capacity and storage question. It represents the larger set of learned weights that make the model's alternatives available. Mixtral's eight experts per MoE layer contribute to that pool even when a particular token selects only two. Hunyuan-Large's sixteen specialized experts contribute to its pool even though a token selects only one of them, while the shared expert is used by every token. The total count therefore tells us that Hunyuan-Large carries a much larger capacity pool, but it does not tell us how much of that pool participates in one token's calculation.

The active count answers a different question: which portion of the model's parameterized computation is selected for a token under the reported routing design? Mixtral's top-2 choice produces its 13B active figure. Hunyuan-Large's shared-plus-top-1 route produces its 52B activated figure. Those are meaningful within their respective designs, but they are not identical denominators. One counts a top-2 route through a homogeneous pool; the other includes a shared path plus a specialized route.

A good comparison therefore keeps two axes visible:

Capacity and storage: how large is the complete parameter pool that must be trained and made available?
Conditional computation: how much of the model's expert path is used for one token, and how is that path assembled?

The two axes answer different operational questions. Mixtral's report uses its lower active count to argue for strong capability with less token-level computation than Llama 2 70B. Hunyuan-Large's report uses 52B activated parameters to frame a model whose total capacity is close to that of a very large dense baseline while comparing favorably on selected tasks. These are different efficiency stories: one emphasizes capability per selected computation, while the other emphasizes the capability of a much larger capacity pool without activating all of it at once.

Neither story makes parameter count causal by itself. The reports differ in training data, tokenizer, optimization, context curriculum, post-training, architecture, and evaluation protocol.

Published by DeepMox

FAQ

Why is Mixtral's 13B active count not the same as a 13B dense model?

Mixtral's 13B figure describes the selected path for a token, while the model still carries a 47B sparse parameter pool whose weights, routing, and placement affect serving.

What does total parameter count tell you?

Total parameter count indicates the size of the learned capacity and storage pool that must be trained and made available; it does not say how much participates in one token's calculation.

Can active parameters explain a benchmark score?

No. A benchmark score also reflects the model's data, architecture, tokenizer, training and post-training, and evaluation protocol, so active count alone is not causal evidence.

Previous8 minMixtral 8x7B and Hunyuan-Large: Shared vs Conditional Routing

Next11 minMixtral 8x7B and Hunyuan-Large: Training to Capability

Mixtral 8x7B and Hunyuan-Large: Active-Parameter Limits

Mixtral and Hunyuan-Large: What Active Parameters Do—and Do Not—Tell Us

Read the two counts as a pair

FAQ