Image: DeepSeek

DeepSeek has released DeepInfer, the software it says runs its own AI service, as free open-source code. The company announced it on X and put the code on GitHub under the Apache 2.0 licence, which lets anyone use it, change it and build paid products on it.

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!

Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.

🔗 https://t.co/JGAik1kgk0

1/4

— DeepSeek (@deepseek_ai) October 20, 2026

DeepInfer is an inference engine. Training builds an AI model. An inference engine is the program that loads the finished model onto chips and turns your questions into answers. That software decides how fast replies come back and how much each answer costs to produce.

DeepSeek says DeepInfer powers DeepSeek-V4.1-Flash "at scale with high throughput and low latency." It says it shared the code to help others "lower compute costs, reduce energy use, and make AI more affordable." DeepSeek also says code for its future models will land in the same project.

For now the engine runs one model. That is DeepSeek-V4.1-Flash-1017, the update DeepSeek made official and released for download last week. Hugging Face lists it at 763 billion parameters.

The hardware list is short. DeepInfer supports NVIDIA "SM100" GPUs and Huawei's Ascend 950 chips. SM100 is NVIDIA's label for its data center Blackwell chips, such as the B200. A GeForce RTX 50 series card uses a different design called SM120, so gaming PCs are off the list.

DeepSeek's speed charts used 4 of those GPUs to read prompts and 64 to write replies. DeepSeek ran those tests itself, and nobody outside the company has repeated them yet. The repo includes a script for anyone who wants to check.

The release also stops short of a full service. The README says this version handles offline batch jobs. It leaves out a KV cache system that stores conversation memory, the setup that splits prompt reading and reply writing across machines, and load balancing. A company would need those pieces to serve live chat.

Here is a detail DeepSeek mentions only in its credits. The README says DeepInfer was "originally built primarily on vLLM," the popular open-source engine, with ideas from SGLang and NVIDIA's TensorRT-LLM. So DeepInfer is a DeepSeek-tuned offshoot of tools developers already know. vLLM and SGLang run many models on many kinds of chips. llama.cpp runs smaller models on laptops and gaming PCs. DeepInfer runs one model on two chip families.

The engine also pulls in kernels DeepSeek published earlier. Kernels are small programs tuned for one chip. FlashMLA and DeepEP came out during DeepSeek's Open Source Week in February 2025, and FlashMLA now has more than 13,000 stars on GitHub. DeepInfer has gathered about 3,400 stars so far.

DeepSeek also released deepseek-recipe, the code that turns text into tokens and back. It says this matches the logic behind its official API. A cloud host running V4.1 Flash should then format answers the same way DeepSeek does.

Who can try DeepInfer

Most readers should keep using the DeepSeek app or API. Any gain for you will come if cloud hosts adopt DeepInfer and cut their prices for V4.1 Flash.

Developers with access to B200 class GPUs or Ascend 950 chips can test it today. You need Linux, Python 3.12, GCC 13 or newer, and CUDA 13 or Huawei's CANN 9.2. Set DEEPINFER_ACCELERATOR to nvidia or ascend and run ./build.sh. Then run python examples/offline_inference.py, which downloads the model from Hugging Face on its first run.

Two settings are worth a look. Batch-invariant mode gives the same output no matter what other requests share the batch. Structured outputs force replies into a JSON schema or a regex pattern. Batch-invariant mode works on NVIDIA chips only for now.