Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning
arXiv:2604.18978v3 Announce Type: replace-cross Abstract: Scaling critic capacity is a promising direction for improving off-policy reinforcement learning (RL). However, recent work shows that larger critics are prone to overfitting and instability in replay-based bootstrapped training. In this paper, we propose using Low-Rank Adaptation (LoRA) as a structural regularizer for critic learning from scratch. Our approach freezes randomly initialized base matrices and optimizes only the correspondi…