Blind former journal editors preferred Claude's paper review to a paid human e… · @willzero · TwKit

@willzero

Blind former journal editors preferred Claude's paper review to a paid human expert's in 26 of 30 pairs. That's from a Harvard working paper by Gary King and Steven Worthington, dated Oct 22. The humans in this test were about as good as reviewers get. rapidPeer, a pre-submission service at Harvard's IQSS, pays outside experts to read the whole manuscript and write at least 1.5 to 2 pages. 88% of surveyed authors called their review high or very high quality. The study covers 124 social science papers and 134 of those reviews. Claude Opus 4.8 ran at max reasoning with the same anonymized paper and the same guidelines, and no tools. GPT-5.5 also judged every pair and picked the AI review in all 132 decided ones. I'd weigh that less, since it's one model grading another. The accuracy check is what convinced me: 5.5% of the AI's claims about the manuscript were judged wrong, against 15.9% for the humans, and the AI cited no fabricated sources. On 44 length-matched pairs the AI still won every one. The paper only tests whether a review helps an author improve a draft. The authors say it doesn't answer how editors should decide what gets published. It also cites ICML 2026, where 22.5% of surveyed reviewers assigned to a no-AI policy said they used AI anyway. If you write papers, I'd run a couple of AI review passes before submitting. Which journal changes its rules first? https://gking.harvard.edu/peerless/

曝光 16 · 评论 0 · 点赞 0 · 书签 0