A Sign Law for Entropy Drift under Sampler Mismatch in RLVR
arXiv:2610.34044v2 Announce Type: replace-cross Abstract: In reinforcement learning with verifiable rewards (RLVR), whether entropy rises or falls can depend on choices that matter little under exact sampling, such as whether over-length responses are masked or counted as failures. We trace this to sampler mismatch: rollouts are drawn with top-k/top-p truncation or at a temperature the learner does not apply. We prove that a constant advantage shift then changes the entropy of a softmax policy,…