MATRIX ยท DeepInfer https://derekwang2002.github.io/ai-intelligence/en/projects/deepseek-deepinfer/ Material changes in AI models, agents, engineering and infrastructure, with primary sources and recommendations. en Mon, 02 Nov 2026 16:56:00 GMT DeepSeek open-sources its production inference engine DeepInfer https://derekwang2002.github.io/ai-intelligence/en/events/ev-20261020-04/ ev-20261020-04-published Tue, 20 Oct 2026 16:04:41 GMT New TRIAL <p>DeepSeek has released DeepInfer, the Apache-2.0 inference engine behind its production services. The first public version supports DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with optimized kernels, parallelism, asynchronous scheduling, deterministic sampling and structured output. It currently provides offline inference and omits the KV-cache, disaggregated-serving and replica load-balancing layers needed for a complete online service.</p><p><strong>Why read: </strong>DeepSeek exposes production inference internals and determinism paths, while the complete serving layer is still missing and should be reviewed before reproduction.</p><p>Recommendation: TRIAL</p><p>Sources: <a href="https://github.com/deepseek-ai/DeepInfer">DeepInfer repository</a></p>