Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity

arXiv:2511.02496v2 Announce Type: replace Abstract: We study latent geometry as an explicit component of representation quality in data-scarce learning. For an encoder (phi), we define (Q_{eta,gamma}(phi)=I(phi(X);Y)-etamathcal C(phi)-gamma d_{int}(phi)), combining task-relevant information with penalties for curvature and intrinsic latent dimension. Thus geometry becomes part of the bottleneck criterion, not only a post hoc diagnostic. Under smooth-manifold, loss-transfer, and estimator-concentration assumptions, we derive non-asymptotic low-label generalization bounds where intrinsic dimension and covering complexity enter explicitly. We characterize the information--geometry frontier and prove empirical-surrogate consistency. The analysis links encoder geometry to learning through latent covering numbers, loss-class entropy, and uniform deviation. We instantiate the theory as exttt{V-GIB}, adding curvature and dimension penalties to variational bottleneck training. Real low-label benchmarks compare exttt{V-GIB} with ERM, VIB, and ablations across (1%)--(20%) label fractions. Results show improved performance and reduced geometric complexity in several regimes, especially FashionMNIST and CIFAR-10, while confirming that no fixed regularizer is universally dominant.

Source: arXiv cs.LG | 2026-07-03