推文

@harden_369741 · 2026-10-13 18:12

Claude Opus 5.5 just scored 52% on the only coding benchmark I actually respect. FrontierCode doesn't ask 'does it pass tests.' It asks 'would a real maintainer merge this PR.' 100 real repo tasks, graded like a human reviewer would. The best model on earth fails half of them. I've had AI write code that passed every test and I still wouldn't merge it. Bloated diffs, files touched for no reason, tests that test nothing. This benchmark finally measures that. The leaderboard is a humility lesson: 9 models from 6 labs crammed between 43-52%. MiMo-V2.6-Pro leads open weights at 45%. GPT-6.1 Sol hits 49% at $0.81 a task, roughly a tenth of Anthropic's cost. The coding race isn't about who's smartest anymore. It's about cheapest per merged PR. When did you last merge AI-written code without rewriting half of it? #AI #coding #LLM

曝光 24 · 评论 0 · 点赞 1 · 书签 0