Too Sharp, Too Sure: When Calibration Follows Curvature

arXiv:2604.20614v1 Announce Type: new Abstract: Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration, curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss--Newton curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.

Related

• Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation
• Predictive Entropy Links Calibration and Paraphrase Sensitivity in Medical Vision-Language Models
• Towards Accurate and Calibrated Classification: Regularizing Cross-Entropy From A Generative Perspective
• Gating Enables Curvature: A Geometric Expressivity Gap in Attention

Source: arXiv cs.LG | 2026-04-23