I compressed the same 201MB log file 6 ways. Two of my assumptions died. The f… · @ReignSwords · TwKit

@ReignSwords

I compressed the same 201MB log file 6 ways. Two of my assumptions died. The file: 201MB of synthetic API logs. Timestamps, IPs, trace IDs. The boring, repetitive text you ship in every release. Seed 42, reproducible. 1. gzip -6: 8.3s → 42.5MB. The tar.gz default. 2. gzip -9: 16.9s → 41.2MB. Double the time for 3% smaller. Trap one. 3. zstd -3: 1.5s → 44.2MB. 11x faster than gzip -9, 7% bigger. Decompresses in 0.4s. My new CI default. 4. zstd -19: 214s → 32.0MB. 22% smaller, 13x the wait. Trap two. 5. xz -6: 158s → 31.6MB. Faster AND smaller than zstd -19. Did not see that coming. 6. xz -9: 388s → 30.9MB. Six and a half minutes for 2% over xz -6. The final boss of diminishing returns. Verdict: gzip -9 is the worst of both worlds. zstd -3 for speed, xz -6 when every byte counts. Copy-paste: tar -I 'zstd -3' -cf logs.tar.zst dir/ I benchmark stuff like this most days — follow along if that's your thing. And be honest: when did you last time your compressor instead of typing -9 from muscle memory? 🔵 Fellow Premium members, let's connect! I post about AI & tech — follow for more 👥

曝光 67 · 评论 5 · 点赞 1 · 书签 0