Infrastructure
Six official repositories that explain why DeepSeek is cheap to run and possible to train. Almost nobody writes about them — but they decide whether you get the same throughput on your own hardware.
Why this layer matters. Most people only ask "which model
is better". But the same model with official kernels versus generic ones can differ 2-3x in
throughput. This layer is that difference.
Efficient Multi-head Latent Attention kernels for Hopper GPUs.
One reason DeepSeek can do long context cheaply.
Expert-parallel communication library for MoE, high-throughput all-to-all.
The MoE bottleneck is communication, not compute. This is the fix.
FP8 GEMM kernels, JIT compiled, tuned for DeepSeek's matrix shapes.
FP8 halves memory, but only with kernels like this.
Fire-Flyer File System — a high-performance distributed file system for AI training.
Data loading is often the training bottleneck; 3FS targets exactly that.
Speculative decoding implementation.
Small model drafts, large model verifies — faster without quality loss.
Official inference engine and serving framework.
Closer to DeepSeek's architecture than generic serving stacks.
Official infrastructure index listing all low-level components.
Start here to understand how DeepSeek was trained.
Bidirectional pipeline parallelism scheduling.
Reduces pipeline bubbles, raises training throughput.
Which one do you need
| Your situation | Look at |
|---|---|
| Deploying a large model, need throughput | DeepInfer, FlashMLA, DeepGEMM |
| MoE model, multi-GPU communication is slow | DeepEP, EPLB |
| Data loading stalls during training | 3FS |
| Want speed without quality loss | DeepSpec |
| Want to understand how DeepSeek trains | open-infra-index |