DeepSeek-V3: The Systems Cost of Scale

Overview

DeepSeek-V3 shows how a 671B-parameter model can activate 37B parameters per token and report 2.788 million H800 GPU hours for official training without making those figures a universal price. The series traces conditional experts, compressed attention state, routing control, DualPipe communication, selective FP8, data and context extension, and post-training from separate bottlenecks to one co-designed system. It explains what each mechanism saves, where work moves, and why strong benchmark results and efficient training still depend on hardware topology, workload, evaluation conditions, and serving scale.

Learning outcomes

DeepSeek-V3's 671B parameters and 37B active parameters describe resident capacity versus a conditional per-token path, not a dense 37B model.
MLA reduces persistent attention state; MTP enriches training supervision and can optionally support speculative decoding, so they address different bottlenecks.
Sparse routing becomes practical only when load balancing, node limits, topology-aware communication, and overlap keep network work from consuming the arithmetic saving.
Selective FP8 and staged data, context, and post-training make efficiency a lifecycle property, while reported training cost and serving scale remain stack-dependent.

Keywords

DeepSeek-V3MoEMLAMTProutingDualPipeFP8KV-cachepretraininginference

Chapters

DeepSeek-V3: Why Active Parameters Aren't Enough

Tracing cost across compute, memory, network, and serving

12 min · Tracing cost across compute, memory, network, and serving

12 min

DeepSeek-V3: How MoE Routing Spends Capacity

Top-eight experts, four nodes, and load control

9 min · Top-eight experts, four nodes, and load control

9 min

DeepSeek-V3: MLA and MTP Target Different Costs

Compressed inference state, richer training signals

10 min · Compressed inference state, richer training signals

10 min

DeepSeek-V3: How Sparse Savings Shift Costs

From active compute to persistent state and network pressure

10 min · From active compute to persistent state and network pressure

10 min

DeepSeek-V3: DualPipe and Topology-Aware Routing

Why overlap cannot erase communication cost

10 min · Why overlap cannot erase communication cost

10 min

DeepSeek-V3: How Selective FP8 Stays Stable

Fine-grained scaling with higher-precision recovery

10 min · Fine-grained scaling with higher-precision recovery

10 min

DeepSeek-V3: From Pretraining to Usable Behavior

How data, context extension, and feedback shape serving cost

12 min · How data, context extension, and feedback shape serving cost

12 min

DeepSeek-V3: What Its Cost Story Proves

Capability gains under a hardware-dependent stack

14 min · Capability gains under a hardware-dependent stack

14 min

Preview every chapter · Unlock full text, audio, and video from chapter 4

Plans

Ch.1-3 FreeCh.4-8 Premium

FAQ

What does DeepSeek-V3's 671B versus 37B parameter count mean?

The 671B figure is resident model capacity, while 37B is the approximate parameter path activated per token; the full system still stores experts, routes tokens, runs shared and dense components, and manages attention.

How does DeepSeek-V3 reduce inference memory and decoding work?

MLA reduces the persistent KV-cache representation, while MTP can propose an additional token for verification; MLA targets retained state and MTP's serving benefit is optional and workload-dependent.

Why is DeepSeek-V3's training cost not its serving cost?

Training and serving stress different resources: the reported 2.788 million H800 GPU-hour figure covers official training, while serving must handle routing, cache state, latency, redundant experts, and a large decoding deployment.

What does DeepSeek-V3 actually prove about efficient large models?

It demonstrates that co-design can make a 671B-parameter model more economical under a matched stack; it does not prove large models are cheap on every hardware platform or workload.