8,192 concurrent sandboxes and ~1.3 million rollouts. Really impressive engine… · @GenAI_is_real · TwKit

@GenAI_is_real

8,192 concurrent sandboxes and ~1.3 million rollouts. Really impressive engineering from @IFM_AI! One thing I find particularly interesting about long-horizon agentic RL is that infrastructure failures can directly affect training correctness. If a sandbox crashes halfway through a trajectory and the system treats it as a failed task, we're effectively penalizing the model for something it never did wrong. At this scale, distinguishing environment failures from actual model failures becomes part of the learning algorithm itself. RL360 also demonstrates why modular RL infrastructure matters. With Miles handling rollouts and optimization, Harbor managing agent environments, SGLang serving inference, and Megatron-LM running training, each component can evolve and scale independently. I'm especially happy to see Miles and SGLang being used together in such a large-scale, real-world RL system. The lessons about asynchronous execution, distributed trajectory management, and sandbox reliability are incredibly valuable for the entire open-source community. Huge congratulations to the IFM team! https://github.com/ifm-ai/RL360

曝光 1244 · 评论 4 · 点赞 9 · 书签 12