Open Problems in Constitutional Preference Reconstruction
DGX agentarXiv:2606.30116v1 Announce Type: new Abstract: Pairwise preference data is widely used for training and evaluating language models (e.g., RLHF), but each datapoint records a choice, not the rationale