The DeepSeek Series: A Technical Overview

DeepSeek-LLM: Establishing the Cost/Scale Frontier

The DeepSeek research series starts with a fundamental scaling question: given a fixed pre-training compute budget, how should you divide it between model parameters and training tokens? Prior work like Chinchilla and GPT-3 offered different prescriptions. DeepSeek-LLM takes a more granular view by defining model scale not as parameter count, but as non-embedding FLOPs/token — the compute spent on the transformer layers excluding embeddings. This yields the relation Compute = FLOPs/token × Data size.

This framing has two immediate payoffs. First, it lets the authors predict training behavior for 7B and 67B models on 2T tokens of bilingual English/Chinese data. Second, it directly addresses the recurring problem of training instability — the sudden, irrecoverable loss divergences that plague large models, particularly those with long contexts or MoE architectures. Their conclusion is that stability is not incidental but a product of what the authors term HPC Co-Design: deliberately engineering the transformer architecture together with the data-center infrastructure it runs on.

The paper also clarifies how data quality shifts the optimal parameter-to-token ratio. Higher-quality data — curated, deduplicated, and information-dense — allows a larger model to extract more signal per token. Lower-quality data hits diminishing returns sooner, meaning a smaller model is a better use of compute.

Practical output includes simple power-law recommendations for scaling batch size and learning rate as compute grows, and the trained DeepSeek-LLM 67B model, which outperforms LLaMA-2 70B on math and coding after supervised fine-tuning and Direct Preference Optimization. The two pillars here — scaling laws and infrastructure-aware training — become the foundation for everything that follows.

DeepSeek-V2: Compressing Attention and Widening the Net

DeepSeek-V2 targets the two dominant bottlenecks in training and serving large models: memory overhead from the KV cache during attention, and the compute cost of dense feed-forward layers. Its two contributions are distinct but complementary.

Multi-Head Latent Attention (MLA) compresses the keys and values into a small latent vector that is shared across attention heads. Rather than storing a full per-head KV cache, only the compressed latent plus a small per-head key component is cached. The effect is a significant reduction in KV-cache memory, which pays off at both long-context training and high-concurrency inference.

On the parameter side, the paper replaces the dense feed-forward network with DeepSeekMoE, a sparse architecture that explicitly separates shared (always-activated) experts from routed experts. This greatly increases total parameter count while capping the compute per token, as each token only activates a small subset of experts. Fine-grained expert segmentation and shared-expert isolation were introduced to address the routing imbalances and knowledge duplication that plague earlier MoE designs.

The two changes are synergistic: MLA handles the long-context memory problem, and MoE handles the wide-network compute problem. Together they make the architecture sparse in both compute and memory, in contrast to the dense baseline. Importantly, the paper validates the design through careful ablation studies that decompose the contribution of each component, strengthening the case that these are not purely ad-hoc fixes but deliberate engineering consequences of the HPC co-design methodology.

DeepSeek-V3: The 671B-Parameter Co-Design Masterclass

DeepSeek-V3 brings the series into the territory of frontier-scale training. The result is a 671B-total-parameter MoE model with 37B activated per token, trained on 14.8T tokens. The core novelty is two-fold: architectural refinement and infrastructure-level co-design at an unprecedented scale.

Auxiliary-loss-free load balancing. MoE models usually need a term in the loss function to encourage uniform token routing across experts. DeepSeek-V3 instead introduces a bias term per expert that is dynamically adjusted during training based on routing load, avoiding the performance degradation that auxiliary losses can introduce. This mechanism runs in the data-parallel group, adding negligible training cost.

FP8 mixed-precision training. The authors validate that FP8 precision can handle the full scale of training, from the forward pass to the backward propagation, for both dense and MoE layers. This does not just cut memory in half — the optimization philosophy extends to a hierarchical scheduling strategy where token dispatch and expert parallelism are carefully interleaved with the communication fabric of the GPU clusters (built on H800s and NVLink/NVSwitch). This co-design lets them train at scale with minimal overhead from idle compute due to communication stalls.

A notable supporting result comes from the model's architectural details: using a deeply (61-layer) transformer with shared output-head parameters lowered KV-cache overhead even further. Combined with MLA from V2, the per-token memory footprint for long-context inference is kept largely in check, making the 671B model practical to serve. The paper also includes post-training alignment via SFT and reinforcement learning, and reports that the model effectively matches or outperforms contemporary dense models like LLaMA-3.1-405B and GPT-4 on standard benchmarks. But the primary contribution is the infrastructure blueprint: training a model of this size on a connected cluster of accelerator nodes is shown to be viable with ordinary hardware, given the right software and memory orchestration.

DeepSeek-R1: Reasoning Without Hand-Crafted Labels

The final paper in this survey pivots from architecture economics to capability acquisition. If V3 established the means to train a 671B model, R1 searches for an answer to: what happens when you let such a model learn to reason at scale, without relying on large amounts of human-annotated reasoning traces?

The key method is large-scale reinforcement learning. The authors demonstrate that reasoning abilities—specifically, long chains of thought with self-correction—can emerge from RL applied to a base model trained on code and math. They report an astonishing result: their “R1-Zero” variant, which skips Supervised Fine-Tuning (SFT) entirely, already provides clear improvements on reasoning tasks. The model learns to spend more time thinking and can revisit its own prematurely drawn conclusions, purely through the RL signal.

However, R1-Zero behaves poorly on readability and “formality” aspects. The full R1 pipeline therefore includes a small amount of cold-start data (less than 1,000 reasoning exemplars) that injects a better formatting prior, followed by RL. The post-training mixes RL for two distinct reward types—accuracy rewards for math/code, and format rewards ensuring chain-of-thought structure—before distilling the R1 reasoning templates into smaller dense models such as DeepSeek-R1-Distill-Qwen-7B with noticeable results.

Two structural observations stand out from this paper:

• It confirms that large-scale RL alone can elicit reasoning in models of a sufficient size, without procedural SFT on chain-of-thought data. The “aha” moment is not written in, but learned, though the RL is brittle when moved out of math/code into tasks with unclear reward signals.
• It carries forward the series' central theme: efficiency is the enabler. R1 scales RL by reusing the compute- and memory-efficient training system of V3, making iterative interactions with the model affordable enough to be explored as a general training tool.

R1 still struggles on non-reasoning tasks, particularly in STEM, where more conventional knowledge-based performance may degrade after RL-only training. The distillation recipes help re-baseline smaller models but do not fully close the gap to the company's own larger, fine-tuned frontier models.

Meta-Observations Across Papers

Read as a unit, the four reports tell a consistent story. The effort each model generation spent on architecture was balanced with an equivalent effort spent on the training infrastructure — nodes, routing, data flow, and gradient synchronization that are all co-designed at each step. The budget-conscious approach central to V2 and V3 makes the large-scale RL of R1 technically and organizationally feasible; had the authors not spent the first two iterations solving the sparse-attention and MoE memory bottlenecks, the cost for the RL loop in the last paper would have been prohibitive.

The breakdown to carry forward:

• Cost/memory optimization is inherent to the design philosophies from the very first paper, where parameter counts and data budgets were pitted against each other mathematically.
• High-performance computing co-design—executed at the hardware, networking, and algorithm levels, not just the PyTorch call graph level—is what allowed a 671B-parameter sparse model to train on commodity H800s.
• Reasoning from pure RL constitutes a new capability frontier that the DeepSeek work demonstrates is accessible to open-source-style R&D budgets, even if not yet a turnkey method for all tasks.

The significance is not so much that any single result is unexpectedly brilliant, but that the engineering culture distilled in these reports signals a maturation of methods for training frontier-scale models under modest budgets—an operating point more open research groups and enterprises can now aim for.

Architectural Levers in DeepSeek-V2

DeepSeek-V2 moves beyond the high-level scale tradeoffs explored in the earlier LLM work, targeting two specific bottlenecks inside the Transformer. First, the attention mechanism’s KV cache—storing Key and Value vectors for every token—consumes significant memory. Second, the feed-forward networks (FFNs) account for the bulk of the FLOPs in the model. The paper introduces two mechanisms to address these issues: Multi-Head Latent Attention (MLA) and DeepSeekMoE, a sparse Mixture-of-Experts design.

Multi-Head Latent Attention

Standard multi-head attention computes separate Query (Q), Key (K), and Value (V) vectors for each head, each with dimensionality tied to d_model. MLA compresses this representation using a low-rank factorization. A single down-projection maps the hidden state to a compressed latent vector shared by keys and values:

$$ \quad \mathbf{c}_{t}^{KV} = W^{DKV}\mathbf{h}_t, \quad \mathbf{k}_{t}^{C} = W^{UK}\mathbf{c}_t^{KV}, \quad \mathbf{v}_{t}^{C} = W^{UV}\mathbf{c}_t^{KV}, \quad $$

Here, c_t^KV is the latent vector, W^DKV is the down-projection, and W^UK, W^UV are up-projections that reconstruct the full K and V heads on the fly. In effect, the model caches only the small latent vector instead of all full-size KV heads, recovering the per-head dimensions only when needed for the attention computation. This reduces the memory footprint of the cache without reducing the expressiveness of the attention heads themselves.

Sparse Experts with Device-Limited Routing

DeepSeekMoE applies a Mixture-of-Experts (MoE) strategy to the feed-forward blocks. Rather than running every token through a dense FFN, the model routes each token to a subset of specialized expert networks. DeepSeek’s implementation is more fine-grained than earlier MoE designs, splitting experts into two categories: shared experts that process universal patterns for every token, and routed experts selected dynamically per token based on affinity scores.

A key practical concern for MoE is communication overhead. If any token can access any expert, the resulting all-to-all data transfer can saturate the interconnect. DeepSeekMoE introduces device-limited routing: for each token, the model first selects a restricted number M of devices whose experts have the highest affinity. Expert selection is then confined to those M devices. This keeps the model scalable across many GPUs without incurring prohibitive cross-device traffic.

Uneven expert utilization is another known failure mode for MoE, where some experts become overused while others stall. To combat this, the training objective includes three complementary balancing losses:

• Expert-level Balance Loss (L_ExpBal): encourages a uniform distribution of tokens across experts to prevent expert collapse.
• Device-level Balance Loss (L_DevBal): ensures the compute workload stays even across devices.
• Communication Balance Loss (L_CommBal): balances incoming and outgoing token routing for each device.

Training Profile and Results

DeepSeek-V2 totals roughly ~236B parameters, with only about 21B activated per token. The model was pre-trained on 8.1T tokens, then fine-tuned with 1.5M supervised instruction samples, followed by reinforcement learning for alignment. The combination of MLA's smaller cache and the sparse activation from the MoE layers yields measurable gains in both training and inference cost, while maintaining stability at scale. This phase makes the benefits of hardware-aware design explicit: shaping the model around the infrastructure—notably, the limited interconnect bandwidth of the H800 cluster—proved essential for the results that followed.

DeepSeek-V3: Systems-Level Co-Design for Sparse Scale

DeepSeek-V3 pushes the MoE architecture further, scaling to 671B total parameters (37B activated). Training on 14.8T tokens completes in under 2.8M H800 GPU hours, an efficiency the authors attribute directly to the co-design of algorithms, frameworks, and hardware.

Lastly, we emphasize again the economical training costs of DeepSeek-V3, summarized in Table 1, achieved through our optimized co-design of algorithms, frameworks, and hardware.
-- DeepSeek-V3 Tech. Report, p.5

Refinements in Attention

The Multi-Head Latent Attention (MLA) mechanism from V2 gets several targeted upgrades aimed at supporting a 128K context window:

• Improved RoPE Handling: V3 introduces a "decoupled shared key" that separates rotational encodings more cleanly, reducing numerical drift over long generations.
• Joint KV Storage: Instead of storing compressed keys and values as distinct entities, V3 merges them into a single compressed representation. This reduces memory traffic, particularly for multi-node inference.
• Layer-Wise Adaptive Cache: The model prunes older KV entries at deeper layers, preventing memory usage from exploding as the context window fills.

These changes are essential for handling the 128K context without exhausting GPU memory, a capacity that lets the model process entire documents in a single pass where many other LLMs cap out at 4K to 32K tokens.

Auxiliary-Loss-Free MoE Routing

On the mixture-of-experts side, V3 abandons the auxiliary-loss penalty used in V2. Instead, each expert accumulates a dynamic bias b_i that adjusts up or down based on load. The routing decision for a token becomes:

s'_{i,t} = s_{i,t} + b_i

This shift yields several advantages:

• No Token Dropping: Overloaded experts no longer cause token loss; the bias mechanism maintains balance without dropping input.
• More Activated Experts: Routing per token increases from 6 to 8 experts, boosting representational capacity.
• Cleaner Optimization: Removing auxiliary losses prevents external penalties from interfering with the core training objective, letting the gating mechanism learn purely from intrinsic signals.

The final feed-forward layer combines a small set of shared experts with up to 8 adaptively selected specialized experts.

HPC-First Framework Design

Training an MoE of this size requires aggressive systems engineering. The authors highlight that they overcame the cross-node communication bottleneck, achieving near-full overlap between computation and communication.

Through the co-design of algorithms, frameworks, and hardware, we overcome the communication bottleneck in cross-node MoE training, achieving near-full computation-communication overlap.
-- DeepSeek-V3 Tech. Report, p.5

Three techniques stand out:

FP8 Mixed Precision: General matrix multiplications (GEMMs) use an FP8 format, halving memory requirements. The reduced numeric range is managed through block-wise scaling (on 1x128 or 128x128 tiles) and periodic promotion to FP32 over short accumulation intervals to avoid overflow.

DualPipe Parallelism: This scheduling strategy reorders pipeline stages so that MoE all-to-all dispatch is hidden behind local matrix multiplications. Cross-node communication over InfiniBand never stalls local compute.

PTX-Level & Warp Specialization: At a level below CUDA, the team tunes warp-level PTX instructions and auto-tunes chunk sizes for dispatch. They also dynamically partition streaming multiprocessors between communication and compute roles, ensuring token routing never idles local GEMM operations.

The result is a training run at 14.8T tokens costing just 2.8M H800 GPU hours—a low figure for the scale involved. The trained model shows strong performance on code, math, and multilingual tasks, outpacing other open-source models of comparable size.

DeepSeek-R1: Pushing Reasoning With Reinforcement Learning

DeepSeek-R1 and DeepSeek-R1-Zero share the same architecture as DeepSeek-V3. The differences between them come entirely from post-training: R1-Zero applies reinforcement learning directly to the V3 base model, while R1 adds supervised steps before, between, and after that RL.

R1-Zero: Pure RL From the Base Model

Earlier DeepSeek releases relied on supervised fine-tuning (SFT) with occasional reinforcement learning. R1-Zero takes the opposite path: no supervised warmup at all, just RL from the base checkpoint using Group Relative Policy Optimization (GRPO). The method works by:

1. Sampling a group of outputs from the old policy ${o_1, ..., o_G}$
2. Scoring each output with a rule-based reward
3. Normalizing each advantage $A_i$ using the group mean and standard deviation
4. Optimizing a clipped objective similar to PPO

The reward is a weighted sum of two components:

• Accuracy Reward — for tasks with verifiable answers, correctness is checked via mathematical solvers for proofs or code execution against test cases
• Format Reward — the model earns credit for structuring output with explicit <think></think> and <answer></answer> markers

For each output, the relative advantage is computed as:

$$ A_i = \frac{r_i - mean(\{r_1, r_2, ..., r_G\})}{std(\{r_1, r_2, ..., r_G\})} $$

where $r_i$ is the reward for that output. The policy update favors higher-reward responses while a clipping function keeps the new policy close to the old one.

The setup is effectively a testing harness around the model, exercised through RL and steered by only accuracy and format signals. That minimal guidance produced observable reasoning behaviors:

• Self-verification — the model checks its own answers
• Extended chain-of-thought — longer, more detailed reasoning traces
• Exploratory reasoning — trying multiple approaches before settling on an answer
• Reflection — questioning its own solutions and adjusting reasoning mid-stream

For researchers, R1-Zero is the most intriguing result of the R1 paper: complex chain-of-thought emerged from raw reward signals alone. But the model had real weaknesses:

• Readability problems — with no human-curated style data, outputs sometimes came out jumbled or mixed multiple languages
• Instability on non-reasoning tasks — math and code solutions could be valid, but simpler Q&A and safety prompts produced awkward responses
• Limited domain — rule-based rewards suit verifiable tasks, but creative or writing tasks need broader reward coverage

The conclusion from the authors: pure RL produces strong reasoning for verifiable tasks, but the model’s usability suffers. That gap led to DeepSeek-R1, an alignment pipeline that adds cold-start data, additional SFT, rejection sampling, and more RL to address R1-Zero’s shortfalls.

R1: Four Stages of Post-Training

Stage 1: Cold-Start SFT

A few thousand curated, human-friendly chain-of-thought examples covering common sense Q&A, basic math, and standard instruction tasks are used for a short SFT pass on the base model. This gives the model:

• Better readability — polished language style and consistent formatting
• Non-reasoning coverage — conversation, factual QA, and creative tasks that rule-based rewards can’t easily judge

Essentially, the seed of user-friendly behavior avoids the brittleness of a zero-SFT approach.

Stage 2: Reasoning-Oriented RL

Large-scale RL follows for math and coding tasks, starting from the cold-start SFT checkpoint. The model retains its language quality while learning verifiable tasks through rule- and tool-based rewards. The same emergent chain-of-thought behaviors appear, without the language mixing or erratic structure of R1-Zero.

Stage 3: Rejection Sampling and Additional SFT

After RL converges, multiple completions are generated per prompt from the RL checkpoint. Automatic verifiers and human checks select the best outputs, forming a new SFT dataset. Standard writing, factual, and safety data from DeepSeek-V3 is folded in to keep the model balanced on non-verifiable tasks. The base model is then fine-tuned on this combined set.

This step further smooths coverage: the best RL answers become training targets, improving both chain-of-thought and clarity.

Stage 4: RL for All Scenarios

A final RL pass runs on diverse prompts — not just math and code, but general helpfulness, safety, and role-playing. Rewards come from rule-based checks and large preference models trained on user preference pairs. The result is a model that:

• Keeps strong chain-of-thought on verifiable tasks
• Aligns with everyday user requests
• Maintains safer, more controlled outputs

Common Threads Across the DeepSeek Line

Though the papers cover different territory — scaling laws, mixture-of-experts, HPC scheduling, and large-scale RL — they share consistent themes:

1. Cost and memory efficiency
   • MLA, MoE gating, device-limited routing, FP8 training, and DualPipe are all designed to maximize hardware utilization under constraints
   • PTX-level instructions and warp specialization hide communication overhead despite limited interconnect speeds on H800 clusters
2. Sparse architecture meets HPC co-design
   • The mixture-of-experts approach evolves across V2 and V3, culminating in a 671B-parameter model trainable on H800 hardware
   • The authors repeatedly argue that HPC co-design is the only practical route to cheap training of multi-hundred-billion-parameter LLMs
3. Emergent reasoning through RL
   • R1 goes beyond supervised training, letting RL signals shape deep chain-of-thought
   • Pre-trained scale combined with targeted post-training yields advanced behaviors like reflection and multi-step verification

Collectively, the DeepSeek series demonstrates that architecture, algorithms, frameworks, and hardware must be co-designed to train LLMs at trillion-token scale. The broader implication: toolchain builders may want to build HPC optimizations into the model compilation path, and AI research teams may benefit from close HPC collaboration starting at the architecture design stage.