Learning Distributionally Robust First-Order Methods for Convex Optimization

arXiv:2605.06585v2 Announce Type: replace Abstract: We propose a distributionally robust approach to learning hyperparameters (e.g., step sizes and momentum coefficients) for first-order methods in convex optimization. Given a dataset of problem instances, we minimize a Wasserstein distributionally robust version of the performance estimation problem (PEP). Our framework unifies two extremes. As the robustness radius vanishes, we recover classical learning to optimize (L2O), and as it grows we recover the PEP worst-case algorithm design problem. We solve the resulting problem with stochastic gradient descent, differentiating through the solution of an inner semidefinite program at each step. We prove high-probability bounds showing that the true risk of the learned algorithm is at most the in-sample L2O optimum plus a slack that shrinks with the sample size, and is no worse than the worst-case PEP bound. On logistic regression, LASSO, and linear programming examples, our learned algorithms achieve strong out-of-sample performance with certifiable robustness, outperforming both worst-case optimal and vanilla L2O baselines.

Related

• A Machine Learning Approach to Two-Stage Adaptive Robust Optimization
• Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling
• Learning the Robustness Mechanism with Bilevel Optimization
• Brenier Meets Adversarial Training: Optimal Transport Geometry for Robust Learning

Source: arXiv cs.LG | 2026-09-30