CM-DPO: Constraint-Margin Direct Preference Optimization for LLM Planning
arXiv:2610.09219v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) treats all constraint violations equally: a 1 budget overshoot and a 1,000 overshoot induce the same training sig