⚡️ Your Qwen3.8-27B GGUF already ships a free draft head. People thought Ollam… · @QwenAIFans · TwKit

@QwenAIFans

⚡️ Your Qwen3.8-27B GGUF already ships a free draft head. People thought Ollama was ~2× faster than llama.cpp on the same file. It wasn't the engine — Ollama just turns MTP on by default (n=4). Stock llama.cpp leaves it off. One RTX 5090, Q4_K_M: off → 73.6 tok/s MTP n=3 → 133.6 tok/s (1.81×) quality stayed at 100% (speculative = lossless) Flag: --spec-type draft-mtp --spec-draft-n-max 2 (with q4_0 KV they hit 136.7 tok/s around 17 GB) One card, one build — your mileage will move. Still: if you're on stock flags, you're leaving free speed on the table. via laxmimerit 👇 https://huggingface.co/Qwen/Qwen3.8-27B/discussions/112

曝光 474 · 评论 4 · 点赞 7 · 书签 0