Separating Geometry from Probability in the Analysis of Generalization

arXiv:2604.19560v1 Announce Type: new Abstract: The goal of machine learning is to find models that minimize prediction error on data that has not yet been seen. Its operational paradigm assumes access to a dataset S and articulates a scheme for evaluating how well a given model performs on an arbitrary sample. The sample can be S (in which case we speak of in-sample'' performance) or some entirely new S' (in which case we speak of out-of-sample'' performance). Traditional analysis of generalization assumes that both in- and out-of-sample data are i.i.d. draws from an infinite population. However, these probabilistic assumptions cannot be verified even in principle. This paper presents an alternative view of generalization through the lens of sensitivity analysis of solutions of optimization problems to perturbations in the problem data. Under this framework, generalization bounds are obtained by purely deterministic means and take the form of variational principles that relate in-sample and out-of-sample evaluations through an error term that quantifies how close out-of-sample data are to in-sample data. Statistical assumptions can then be used extit{ex post} to characterize the situations when this error term is small (either on average or with high probability).

Related

• Variational Autoencoder Domain Adaptation for Cross-System Generalization in ML-Based SOP Monitoring
• Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
• UniPROT: Uniform Prototype Selection via Partial Optimal Transport with Submodular Guarantees
• Loss-Driven Bayesian Active Learning

Source: arXiv cs.LG | 2026-04-22