The Cost of a Large Model Is a Systems Question
DeepSeek-V3 is a useful test of a deceptively simple question: how can a language model become much larger without making every training step and every generated token proportionally more expensive? The subject here is not Mixture-of-Experts models in the abstract, but DeepSeek-V3 and the way its architecture, training software, numerical methods, data, and deployment plan fit together. The practical judgment this series supports is whether a model's reported efficiency comes from a durable design principle or from a cost figure that only makes sense inside one carefully matched stack.
The headline is deliberately striking. DeepSeek-V3 contains 671 billion parameters, yet the report says that 37 billion are activated for each token. It was pretrained on 14.8 trillion tokens, then extended to longer contexts and post-trained for instruction following and reasoning. The authors report 2.788 million H800 GPU hours for those official training stages, which they convert to $5.576 million at an assumed rental price of $2 per GPU hour. At the same time, their evaluations place the model among the strongest open-source systems of the period and close to leading closed models on many tests.
Those numbers do not describe one miracle. They describe several different kinds of work being controlled at once. A large model must compute a transformation for each token, retain information from earlier tokens during generation, move routed activations across machines, represent numbers precisely enough to train without instability, learn from enough useful data, and produce answers whose length also affects serving cost. DeepSeek-V3 addresses these pressures with different mechanisms. Understanding the result means keeping those mechanisms separate long enough to see what each one contributes, then reconnecting them into one operating model.
A large model with a small per-token path
A parameter count describes the capacity stored in a model, not necessarily the amount of that capacity used for every token. DeepSeek-V3 uses a Transformer with a Mixture-of-Experts feed-forward design. In the layers that use MoE, a token is processed by a shared expert and a selected subset of routed experts rather than by every routed expert. The published configuration has 256 routed experts, with eight selected for each token. This is how the model can hold a large pool of learned transformations while keeping the active path much smaller than the resident model.
That distinction matters, but it is easy to overread it. The 37B active-parameter figure is an architectural accounting of the per-token path. It does not mean that the other parameters disappear, that routing is free, or that end-to-end latency is equivalent to running a dense 37B model. The system still has to store the experts, decide where a token goes, move it to the appropriate devices, and combine the results. Attention has its own state to manage. Batches and hardware utilization determine whether theoretical sparsity becomes observed throughput.
The distinction also clarifies what the training-token count means. The 14.8 trillion tokens are not a measure of model size; they are the examples from which the parameters are adjusted. During training, each example invokes forward and backward computation, communication, memory movement, and optimizer updates. During inference, the model repeatedly applies its weights to new tokens and retains state from the preceding context. A model can therefore be economical in one phase and demanding in another. The relevant question is always: which work is being reduced, and which work remains?
The report's cost table is precise about its own scope. It assigns 2.664 million H800 GPU hours to pretraining, 119,000 to context extension, and 5,000 to post-training. The total is an official-training account, not a full accounting of the earlier experiments that led to the architecture, data pipeline, or numerical recipe. The dollar conversion assumes a particular rental price. It also says nothing by itself about electricity, engineering labor, network infrastructure, or the cost of serving a model to users after training. The figure is valuable because it makes the authors' claim concrete; it is misleading only if it is treated as a universal price for building any model of this size.
The capability claim has a similar shape. DeepSeek-V3 reports strong results on knowledge, code, math, and long-context tests, including 88.5 on MMLU, 90.2 on MATH-500, and 42.0 resolved issues on SWE-Bench Verified in the chat comparison. Those results make the efficiency question consequential: the model is not being presented as a cheap system that accepts a large quality loss. But benchmark scores depend on prompts, output limits, evaluation dates, judge models, and implementation details. They establish what the authors measured under stated conditions, not a timeless guarantee of general performance.
Cost appears in different places
The most useful mental model is to treat DeepSeek-V3's cost as a set of bottlenecks rather than a single number. Conditional experts primarily target arithmetic and stored capacity. Multi-head Latent Attention targets the information that autoregressive inference must retain. Routing and distributed scheduling target the movement of activations between devices. FP8 targets compute density, activation storage, and communication volume, but only if numerical error is controlled. Data construction and post-training determine how much capability the training budget produces and how many tokens a useful answer consumes later.
Attention illustrates why active parameters are not enough. In autoregressive generation, the model does not start from nothing at every step. It retains information about previous keys and values so that the next token can attend to the existing context. As the context grows, that retained key-value state becomes a memory concern. DeepSeek-V3's MLA architecture compresses the key and value information into a lower-dimensional latent representation and keeps a separate positional component. The point is not to make the model smaller in the abstract; it is to reduce the state that must be cached for continued generation. The same architecture also compresses queries to reduce activation memory during training.
Sparse experts create a different pressure. If a token can use only some experts, those experts may be distributed across many devices. The arithmetic saved by not running every expert can be offset by waiting for tokens to cross the network or by leaving some devices underused. A router that always chooses the same experts creates a computational hotspot; a router forced into perfectly equal choices for every sequence may lose useful specialization. DeepSeek-V3's later design choices make sense only against this tension between conditional capacity and communication.
Precision introduces another boundary. Lower-precision arithmetic can reduce memory movement and increase compute density, but outliers and accumulation error can turn a nominal speedup into an unstable training run. The authors therefore do not describe the model as simply “trained in FP8.” Their framework uses FP8 for most dense matrix multiplications while retaining higher precision for sensitive components and for selected stored quantities. The saving depends on where reduced precision is safe and on whether the hardware can accumulate and rescale the products accurately enough.
Finally, the useful output is part of the operating cost. A reasoning procedure that improves an answer by generating more intermediate text may increase the number of forward passes needed per request. Context extension adds a separate training phase rather than arriving automatically with the base model. Post-training also has to balance accuracy, style, and response length. These are not peripheral concerns. They determine whether an efficient training recipe produces an efficient service.
The diagram's connections are not a claim that each component contributes an independently measurable share of the dollars. They show the levels at which the report addresses different constraints. Architecture changes how much computation is conditional. Runtime changes whether that conditional computation is blocked by communication. Precision changes how much data and arithmetic the hardware can handle. Data and post-training change how much capability is extracted and how much generation work is required later.
The co-design bargain
DeepSeek-V3's central lesson is that these levels reinforce one another. A fine-grained expert design is useful only when routing remains balanced and tokens can reach experts efficiently. Compressed attention state matters most when long contexts make retained memory expensive. FP8 helps only when quantization, accumulation, and high-precision exceptions preserve a stable optimization process. Training more data is worthwhile only when the system can process it at a reasonable rate and the post-training pipeline can turn the learned representations into reliable behavior.
This is why the report repeatedly joins algorithm, framework, and hardware. The model is trained on a 2,048-H800 cluster using pipeline and expert parallelism, with custom communication paths between InfiniBand and NVLink. The runtime is not an invisible container around the neural network; it shapes the routing limit, the schedule, the memory layout, and the feasible expert granularity. In the authors' account, training remained stable without an irrecoverable loss spike or rollback. That is an important observation about this run, but it is not a promise that the same recipe will be stable on different hardware or with different data.
The bargain also has an uncomfortable side. A communication path can be hidden under computation without being free. A low-precision tensor can be cheaper to store while still requiring higher-precision safeguards. A model can be inexpensive to pretrain while requiring a large, carefully partitioned deployment unit. The report acknowledges that its recommended inference setup may burden small teams and that its use of GPU streaming multiprocessors for communication leaves room for hardware improvement. These qualifications strengthen the efficiency claim because they identify where the cost has moved instead of pretending it has vanished.
The right conclusion at this point is therefore narrower and more useful than “large models are cheap.” DeepSeek-V3 demonstrates that a very large model can be made substantially more economical when its conditional computation, retained state, communication pattern, numerical format, data schedule, and deployment environment are designed together. It does not show that parameter count no longer matters, that a training bill transfers unchanged to inference, or that the same result follows from adopting one component in isolation.
That distinction sets up the first concrete dependency in the system. If 671 billion parameters are available but only a fraction is activated for each token, the next question is not merely how the router chooses. It is how shared capacity, routed capacity, balancing, and network limits interact without turning the unused model into an operational liability. DeepSeek-V3's answer begins with the way it spends parameters.