Wrong Votes, Right Gradients: Why Majority-Vote Rewards Track Verifiable RL
arXiv:2610.20898v1 Announce Type: cross Abstract: Reinforcement learning with self-generated labels, such as majority vote over a model's own samples, often recovers most of the gain of training with verifiable rewards. This is puzzling from the usual prompt-level view, in which majority vote only returns a model's modal answer and cannot add information about correctness. We argue that the relevant unit is the batch-level update. Because parameters are shared, a prompt's own gradient accounts…