👀 Your 8x RTX PRO 6000 "ceiling" on Qwen3.8-27B was not the GPUs. SGLang print… · @QwenAIFans · TwKit

@QwenAIFans

👀 Your 8x RTX PRO 6000 "ceiling" on Qwen3.8-27B was not the GPUs. SGLang printed this at every boot: max_running_requests capped to 12 by the mamba state cache. They were dispatching 256. Engines ran 12 each. Forever. The trap fix: raise --max-mamba-cache-size and KV pool falls from 349K to 45K tokens. Dead end. The real flag: --mamba-ssm-dtype=bfloat16 c192 went 3,833 to 7,882 tok/s (+106%). Same GPU. Same model. One dtype change. Still leaving that log line unread? via HelixML 👇 https://helix.ml/blog/the-ceiling-was-a-state-cache

曝光 276 · 评论 0 · 点赞 4 · 书签 0