How to post-train on a surrogate: Envelope sampling mitigates reward hacking
arXiv:2610.11281v1 Announce Type: cross Abstract: Large language models (LLMs) are commonly post-trained against LLM judges and other cheap surrogates because the true reward, such as human preference