Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices

arXiv:2608.03368v1 Announce Type: new Abstract: For n unit vectors x_1,ldots,x_n in R^d, we study the continuous ReLU derivative Gram matrix H, whose entries are obtained by averaging pairwise gated inner products over a standard Gaussian direction. Writing Delta_pm := min_{i neq j} min{ |x_i-x_j|2, |x_i+x_j|2 } for their projective separation, we prove the universal dimension-free lower bound lambda{min}(H) = Omega( Delta_pm/sqrt{log n} ) . Conversely, we construct worst-case families satisfying the matching upper bound lambda{min}(H) = O( Delta_pm/sqrt{log n} ) , showing that this rate is tight up to universal constants.

Source: arXiv cs.LG | 2026-08-05