Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression

arXiv:2505.11143v2 Announce Type: replace-cross Abstract: Sparse linear regression is a fundamental tool in data analysis. However, traditional approaches often fall short when covariates exhibit structure or arise from heterogeneous sources. In biomedical applications, covariates may stem from distinct modalities or be structured according to an underlying graph. We introduce extit{Neural Adaptive Shrinkage} (Nash), a unified framework that integrates covariate-specific side information into sparse regression via neural networks. Nash adaptively modulates penalties on a per-covariate basis, learning to tailor regularization without cross-validation. We use a extit{split variational empirical Bayes} algorithm that decouples prior learning from posterior inference, reducing the M-step from O(p) neural-network passes per sweep to a single batched pass, a extit{74 to 106x wall-clock speedup} over previously proposed coordinate ascent CAVI for p between 10^2 and 10^4. Experiments on real data demonstrate that Nash improves accuracy and adaptability over existing methods.

Source: arXiv cs.LG | 2026-05-19