⚡️ Qwen3.8-27B AWQ on a 32GB 4080 SUPER in SGLang: about 6 tok/s. Same GPU, ll… · @QwenAIFans · TwKit

@QwenAIFans

⚡️ Qwen3.8-27B AWQ on a 32GB 4080 SUPER in SGLang: about 6 tok/s. Same GPU, llama.cpp GGUF: about 65 tok/s. Ten times slower. The only stable SGLang path is eager decode with CUDA graphs off. Graph capture hangs. Native MTP fails to allocate any request slot. If you're on Ada and picking an engine for this model, that 10× matters. Anyone cracked a fast AWQ path on SM89 yet? via sgl-project/sglang 👇 https://github.com/sgl-project/sglang/issues/36048

曝光 250 · 评论 0 · 点赞 4 · 书签 2