💥 Qwen3.8 MTP can double your decode for free. One wrong KV flag and it turns … · @QwenAIFans · TwKit

@QwenAIFans

💥 Qwen3.8 MTP can double your decode for free. One wrong KV flag and it turns into 224-token spam. RTX 5090 + vLLM NVFP4. MTP K3 took them from 51.5 to 107.6 tok/s. GSM8K stayed 194/200 correct either way, same 6 misses, McNemar p=1.0. Zero quality cost. The flag that breaks it: any kv-cache-dtype override. This checkpoint ships fp8 KV calibration scales. Override them and MTP collapses into single-token loops that short-prompt smoke tests still miss. Leave KV at fp8, turn on MTP with 3 speculative tokens, and send preserve_thinking false on multi-turn. via Murai-Labs 👇 https://github.com/Murai-Labs/qwen3.8-27b-rtx5090

曝光 208 · 评论 1 · 点赞 5 · 书签 5