👀 Same disk-ish size. Wildly different long-context bill. Qwen3 32B Q4_K_M: bu… · @QwenAIFans · TwKit

@QwenAIFans

👀 Same disk-ish size. Wildly different long-context bill. Qwen3 32B Q4_K_M: bumping 8k→32k context costs about +6 GiB VRAM. Qwen3.8 27B Q4_K_M: that same jump costs about +1.5 GiB. Why: Qwen3.8 only keeps full KV on 16 of 64 layers. The other 48 are linear-attention with a fixed state. Planning a long chat on one 24 GB card? Architecture can beat raw param count. Caveat: one request, FP16 KV. via Gosha / http://gpu.nz 👇 https://gpu.nz/en/guides/llm-vram-table

曝光 206 · 评论 1 · 点赞 1 · 书签 1