Users as Annotators: LLM Preference Learning from Comparison Mode
arXiv:2510.13830v2 Announce Type: replace-cross Abstract: Pairwise preference data have played an important role in the alignment of large language models (LLMs). Each sample of such data consists of