Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation

arXiv:2610.20903v1 Announce Type: new Abstract: We study bounded regression under model misspecification, evaluating a function f by its L_1(P_X) distance to the conditional mean eta(x)=E[Ymid X=x], rather than the standard L_2(P_X). This L_1 metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call sign-betting regression. Our main results show that sign-betting regression achieves high-probability L_1 oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability L_1 oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.

Source: arXiv cs.LG | 2026-10-16