💥 You don't need to pick 4-bit or 8-bit anymore. ms-swift EntroPack lets you s… · @QwenAIFans · TwKit

@QwenAIFans

💥 You don't need to pick 4-bit or 8-bit anymore. ms-swift EntroPack lets you set any bpp for Qwen3.8-27B LoRA — fractional too. Same LaTeX OCR recipe on 2×H20, only quant changes: • 2.0 bpp → 10.57 GiB / 19.43 GiB peak (~5.8 GiB under NF4) • 4.0 bpp → 7.19% weight error (NF4: 9.29%) • All four runs hit >99.5% val token acc Catch: CUDA-only for now, and decode can lag on token-by-token gen. via ms-swift docs 👇 https://github.com/modelscope/ms-swift/blob/main/docs/source_en/BestPractices/Flexble-Quantization-With-Entropack.md

曝光 463 · 评论 1 · 点赞 5 · 书签 3