Learning Pareto Stationary Fronts via Single-Pass Backpropagation

arXiv:2610.06397v1 Announce Type: new Abstract: We propose MOSEL (Multi-Objective Stackelberg Efficient Learning), a framework for a posteriori multi-objective optimization (MOO) in deep neural networks that recovers a full front of Pareto stationary solutions at the computational cost of standard single-objective training. MOSEL reformulates the problem as a bilevel optimization problem that leverages network modularity to decouple representation learning from objective-preference alignment. Casting the bilevel optimization problem as a Stackelberg game enables solving the original a posteriori MOO problem in a single forward-backward pass. As a result, MOSEL matches the time and memory efficiency of standard single-objective training while enabling scalable Pareto stationary front learning. Empirically, MOSEL uncovers diverse and optimal Pareto frontiers in strongly conflicting settings (e.g., fairness-accuracy). Remarkably, even in weakly conflicting regimes such as multi-task learning, it consistently converges to solutions closer to the utopia point, outperforming both standard single-objective training and specialized multi-task learning methods. These results highlight the broader potential of a posteriori MOO learning as a pathway to efficiently learn more diverse and robust representations, ultimately improving generalization.

Related

• A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets
• Population-Free Pareto Tracking for Sample-Efficient Multi-Policy MORL
• Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning
• Fed-Equilibrium Framework for Topological Pareto Control in Robust and Fair Clinical Federated Learning
• Joint Optimization of Training and Inference in Federated Edge Learning via Constrained Multi-Objective Deep Reinforcement Learning

Source: arXiv cs.LG | 2026-10-06