⚡️ Same AMD box, 4.6× more chat memory for Qwen3.8-27B. vLLM just shipped Octa… · @QwenAIFans · TwKit

@QwenAIFans

⚡️ Same AMD box, 4.6× more chat memory for Qwen3.8-27B. vLLM just shipped Octave KV (3-bit). On 4× RX 7900 XTX it held 4.6× the cached tokens of a 16-bit cache. Model's own context cap still applies. Caveat: author numbers on ROCm; AMD GPUs only for now. via vLLM PR 👇 https://github.com/vllm-project/vllm/pull/59774

曝光 768 · 评论 4 · 点赞 10 · 书签 0