DeepSeek open-sources DeepInfer, the framework behind its production service
TL;DR
- DeepSeek published DeepInfer on GitHub on 16 October under the Apache License 2.0; by 25 October the repository had 22,315 stars and a Hacker News thread at 2,347 points.
- It serves one model, DeepSeek-V4.1-Flash, on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with speculative decoding, batch-invariant inference, deterministic sampling and structured outputs.
- DeepSeek says a complete service still needs a KV cache system, prefill and decode disaggregation and load balancing, none of which are in this release.
Read the full story
Sign in with your email to read AI News. It’s free.