推文

@GenAI_is_real · 2026-10-15 04:39

Most successful RL recipes start with a cheap verifier. Code passes tests, math answers match, and the reward is relatively straightforward. Professional knowledge work rarely gives you that luxury. That's what makes Ultramarine 1 Flash so interesting to me. @bertgodel and the team built 88 types of professional environments across finance, law, consulting, and operations, with task-specific rubrics evaluating the actual work products. Their reported results are impressive: five people, ~$300K in post-training, and improvements over the base model on all six knowledge-work benchmarks. I work on agentic RL, and this makes me think that non-verifiable is not always an intrinsic property of a task. Sometimes it just means we haven't invested enough engineering effort into making the outcome verifiable. A spreadsheet has formulas we can check. A diligence memo has sources, assumptions, and decisions we can audit. Professional judgment is harder to quantify, but not necessarily impossible to evaluate systematically. Of course, the difficult part is ensuring that models learn genuine professional competence rather than simply optimizing for synthetic rubrics. I'm particularly interested in how well these gains transfer to messy, real-world customer workflows. Still, this is an exciting direction. The economics of post-training may be changing faster than we realize, especially when a small team can build a specialized model around a well-designed training and evaluation environment. Huge congratulations to @bertgodel and the team. Looking forward to the open weights and independent evaluations!

曝光 526 · 评论 2 · 点赞 10 · 书签 9