From Approximation Rates to Loss-Landscape Barrier Decay in Shallow ReLU Networks

arXiv:2602.17596v2 Announce Type: replace Abstract: We study pathwise connectivity of sublevel sets for one-hidden-layer ReLU networks with constrained first-layer weights and an ell_1 penalty on the output layer. The data term is assumed convex and globally Lipschitz in the scalar logit. We first give a finite-width construction that connects any two points of a common sublevel through a path controlled by a loss-consistent compression functional and a first-order perturbation term. The proof replaces the quadratic perturbation estimate in the Freeman--Bruna mechanism by a direct Lipschitz bound. Positive homogeneity is then used in a direction that is compatible with the penalty: every active atom is moved monotonically from the unit ball to the unit sphere while its output coefficient is reduced. Sphere covering and cluster merging consequently give O(m^{-1/(n-1)}) fixed-level thickening for nge2, while the one-dimensional two-ray dictionary gives exact connectivity for every mge4. We also prove internally that the regularized approximation values satisfy e(l)-e_infty=O(l^{-1/2}). More generally, a rate O(l^{-s}) transfers to a near-optimal barrier rate O(m^{-s/((n-1)s+1)}); under the standing assumptions, this yields the explicit rate O(m^{-1/(n+1)}). A theorem-aligned finite-distribution experiment complements the analysis. The primary Huber run yields a maximal best certified upper gap 1.66imes10^{-5} over 720 recorded pairs at widths mge16; a matched binary-cross-entropy rerun and a 720-endpoint dense-representation stress test probe loss robustness and the active cluster-merging mechanism.

Source: arXiv cs.LG | 2026-08-14