DeepSeek just open-sourced the engine that runs its entire production stack. D… · @ReignSwords · TwKit

@ReignSwords

DeepSeek just open-sourced the engine that runs its entire production stack. DeepInfer: the inference framework behind 155 trillion tokens a day per cluster. 11,000+ stars in 24 hours. Funny part: in April 2025 this same team said they'd never release it. Why it's actually interesting: 1. The famous kernels in one install — FlashMLA, DeepEP, DeepGEMM, TileKernels. 2. DSpark: speculative decoding with draft tokens, verification scheduled by confidence. 3. SWA bounded replay: replay only a bounded suffix instead of the whole prefill. 4. Batch-invariant inference + deterministic sampling. Same input, same output, whatever the batch mix. Debugging gold. The cold water: the README lists three things "required for a complete inference service" that are NOT in this release — the KV cache system, prefill/decode disaggregation, load balancing. They open-sourced the engine, not the chassis. Credit where due: "originally built primarily on vLLM." At least the genealogy is printed on the page. Who it's for: if you serve MoE models at scale on SM100 or Ascend 950 — yes, only the two newest chips, no H100 — this is a serious reference implementation. If you run vLLM on one GPU, it's homework, not a migration. Verdict: the most substantive DeepSeek release since the V4.1-Flash weights. Not because of what's inside — because of what's missing. Open-source generosity is a spectrum, and the missing KV cache system marks exactly where the line is. I post the infra stuff that actually moves your bill — follow for more. Would you trust an "open" stack where the cache and the load balancer are the parts they kept? 🔵 Fellow Premium members, let's connect! I post about AI & tech — follow for more 👥

曝光 50 · 评论 6 · 点赞 0 · 书签 0