The new THReD Agent Benchmark evaluates an electronic health record preprocess… · @ktwu01 · TwKit

@ktwu01

The new THReD Agent Benchmark evaluates an electronic health record preprocessing assistant with 141 tasks covering simple and multi-step transformations, failure recovery, requests where the agent should not act, schema changes, and prompt injection hidden in data. It uses synthetic health-record tables, expert reference pipelines, and outcome checks rather than patient data. If you are evaluating assistants that prepare clinical datasets, this artifact tests operational safety and recovery alongside successful preprocessing. It offers a way to compare context configurations without exposing patient records and avoids treating task completion alone as sufficient evidence of readiness

曝光 29 · 评论 0 · 点赞 0 · 书签 0