Everyone cheered "DeepSeek open-sourced its inference engine." I cloned the re… · @ReignSwords · TwKit

@ReignSwords

Everyone cheered "DeepSeek open-sourced its inference engine." I cloned the repo and read the actual code. Here's what they gave away — and what they quietly kept. 1. The repo is real and it's big: 342 files, ~55K lines of Python, plus CUDA kernels. Created Oct 16, announced Oct 20. 17.5K stars in two days. 2. What you get: the scheduler, speculative decoding (5 draft tokens by default), SWA bounded replay, structured outputs, batch-invariant inference. The full serving engine behind V4.1-Flash. 3. What you don't get: their own README lists it under "not included in this release" — a KV cache system, prefill/decode disaggregation, load balancing. The parts that turn an engine into a *service*. 4. The smallest "efficient" deployment, per their own Q&A doc: 64 GPUs for prefill + 64 for decode. 128 Blackwell GPUs. The code is free; the power bill is not. 5. I tried to run it. `import deepinfer` works on plain Python 3.12. One import deeper it dies on `import torch` — no GPU on my box, obviously. This thing only runs on SM100s and Ascend 950s. My takeaway: this might be the most honest open-source drop in months. No marketing fluff — the README tells you exactly what's missing. But "open source" here means "free if you already own a data center." The engine is free. The transmission stays in Hangzhou. If you like this kind of code-level teardown, stick around — I do one every day. Real question: would you rather have the weights or the serving stack? I'd take the stack. Weights get copied in a week; the serving tricks took years. 🔵 Fellow Premium members, let's connect! I post about AI & tech — follow for more 👥

曝光 74 · 评论 7 · 点赞 1 · 书签 1