How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
arXiv:2605.24749v1 Announce Type: cross Abstract: Reward modeling is not only a prediction problem: in KL-regularized policy optimization, the learned reward is exponentiated to define the deployed po