Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One

arXiv:2607.04597v1 Announce Type: new Abstract: In this paper, we study the universal approximation property of residual neural networks, and obtain some new results. For input and output dimensions d_x and d_y, and LeakyReLU, ReLU, ReLU-like activation functions, the upper and lower bounds of the block width are established. To achieve L^p approximation (1leq p <+infty) on any compact domain, we show that the exact minimum block width is max{d_x,d_y} when the inner width is 1. Furthermore, we show that residual neural networks with block width min{d_x+d_y, max{2d_x+1,d_y}} can achieve uniform approximation on any compact domain under the constraint that each residual branch has inner width 1. Besides, for any activation function family, we prove that residual neural networks with block width less than max{d_x, d_y} cannot approximate all target functions, both in the L^p sense and the uniform sense, regardless of inner width.

Source: arXiv cs.LG | 2026-07-07