Limitations of Learning Tanh Neural Networks with Finite Precision

arXiv:2606.11104v1 Announce Type: new Abstract: We investigate limitations of learning anh neural networks from point evaluations under finite-precision computations and L^p accuracy guarantees, building on Berner, Grohs, and Voigtlander (2023). Our approach is based on a novel construction of sharply localized bump functions via iterated anh activations. Using this mechanism, we show that, in a finite-precision setting, no adaptive randomized algorithm based on m samples can achieve a convergence rate higher than the Monte Carlo rate O(m^{-1/p}) in the L^p norm, unless the sampling budget grows exponentially with the size of the network parameters and architecture. The results reveal fundamental limitations imposed by finite precision on the learnability of classes containing localized bump functions, extending previous results for ReLU networks to the anh setting.

Source: arXiv cs.LG | 2026-06-10