Benchmark Radar daily update: The new Cost Per Correct benchmark ranks large l… · @ktwu01 · TwKit

@ktwu01

Benchmark Radar daily update: The new Cost Per Correct benchmark ranks large language model agents by dollars spent per correct answer, not accuracy alone. Its five easy tasks use deterministic regular-expression checks rather than another model as judge, so the measured trade-off is directly between correctness and run cost. This is a small version 1.0 release, not evidence of a broader trend. If you are selecting an agent for high-volume, straightforward work, this benchmark can expose cases where a slightly less accurate but much cheaper model delivers more correct answers for the same budget. Its current task range is too narrow for complex-agent selection, but the metric can be applied to internal workloads

曝光 49 · 评论 1 · 点赞 0 · 书签 0