Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
arXiv:2610.21404v1 Announce Type: cross Abstract: To improve task performance while encouraging desirable response properties, LLM post-training often combines a primary reward (e.g., correctness) with auxiliary rewards (e.g., brevity or efficiency). However, how these rewards are aggregated can critically affect training outcomes. We show that symmetric aggregation, which treats reward channels equally, can induce a failure where aggregate reward improves while primary-task performance degrade…