Uncertainty-Normalized Margins for Direct Preference Optimization
arXiv:2609.38647v1 Announce Type: new Abstract: Direct preference optimization (DPO) models binary preferences through a Bradley-Terry model with a common noise scale, without explicitly accounting fo