DeepSeek has published Apache-2.0 DeepInfer code, hardware boundaries, offline examples and reproduction scripts, while explicitly listing components still missing for a complete online service.
DeepSeek open-sources its production inference engine DeepInfer
2026-10-20Verified announcementTRIAL
DeepSeek has released DeepInfer, the Apache-2.0 inference engine behind its production services. The first public version supports DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with optimized kernels, parallelism, asynchronous scheduling, deterministic sampling and structured output. It currently provides offline inference and omits the KV-cache, disaggregated-serving and replica load-balancing layers needed for a complete online service.
What this means for you
For teams self-hosting DeepSeek models, DeepInfer exposes production-oriented scheduling and determinism mechanisms that were previously harder to reproduce from model weights alone. The hardware and service boundaries are narrow, so it is a trial target rather than a drop-in replacement for vLLM or SGLang.
Evidence & related material
Technical details & scores
| License | Apache-2.0 |
|---|---|
| Supported Model | DeepSeek-V4.1-Flash |
| Hardware | NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs |
| Included | optimized kernels, Attention DP / balanced CP, MoE EP + EPLB, asynchronous scheduling, deterministic sampling and structured output |
| Missing For Online Service | KV cache system, prefill/decode disaggregation and replica load balancing |
| Benchmark Scope | author-reported on 4 SM100 GPUs for prefill and 64 for decode |
| Time Basis | repository was publicly linked by 2026-10-20T09:08:54Z; the official repository does not expose an exact public-release timestamp |
Technical impact 4
Engineering value 4
Adoption signal 4
Maturity 2
Verification cost 4
Risk 3