DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
arXiv:2605.08441v2 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) generates hundreds of thousands of tokens per training step, with rollout generation dominating the computational cost. The overall token budget can be controlled along two main dimensions: (i) deciding which prompts to allocate rollouts to, and (ii) deciding how long each rollout should be. Prior work has generally controlled only one of these dimensions at a time. We show that joint…