Conditional regression for the Nonlinear Single-Variable Model

arXiv:2411.09686v4 Announce Type: replace-cross Abstract: Regressing a function F on R^d without incurring the statistical and computational curse of dimensionality requires exploitable structure. Compositional models F=firc g in which g has a low-dimensional range include classical single- and multi-index models as well as certain neural networks; while the case of linear g is well understood, substantially less is known for nonlinear g. We study the model F(X)=f(Pi_gamma X), where Pi_gamma is the closest-point coordinate associated with an unknown regular curve gamma, and f is an unknown one-dimensional link function. The predictor X need not be intrinsically low-dimensional and may have full-dimensional variation throughout a tubular neighborhood of the curve. We construct a nonparametric estimator based on response slicing, local principal component analysis, data-adaptive slice assignment, and one-dimensional local polynomial regression. Under coarse monotonicity of f and sufficient variation normal to the curve relative to the observational noise and the coarse-monotonicity scale, the estimator attains, up to logarithmic factors, the minimax-optimal one-dimensional mean squared rate down to an explicit geometry- and noise-dependent saturation level. When the normal-variation condition is removed, we prove a complementary guarantee for the wide-slice regime. The estimator can be constructed in time O(d^2nlog n), and the constants and sample-size thresholds in our bounds depend at most polynomially on the ambient dimension d.

Source: arXiv cs.LG | 2026-08-25