Learning from All-Failed Groups via First-Visit Coverage in Long-Horizon Agentic Tasks
arXiv:2607.22724v2 Announce Type: replace-cross Abstract: Group-based policy optimization has been increasingly used to train large language model (LLM) agents from sparse outcome rewards by comparing trajectories or steps within a group. However, on difficult long-horizon agentic tasks, we found this comparison suffers from a sampling imbalance: repeated or low-effect actions dominate the high-probability region of the policy, leaving useful state-changing actions under-sampled. This imbalance…