Diffusion-Guided Feature Selection via Nishimori Temperature: Noise-Based Spectral Embedding

arXiv:2604.24692v1 Announce Type: new Abstract: We propose Noise-Based Spectral Embedding (NBSE), a physics-informed framework for selecting informative features from high-dimensional data without greedy search. NBSE constructs a sparse similarity graph on the samples and identifies the Nishimori temperature eta_N the critical inverse temperature at which the Bethe Hessian becomes singular. The corresponding smallest eigenvector captures the dominant mode of an intrinsically degree-corrected diffusion process, naturally reweighting nodes to prevent hub dominance. By transposing the data matrix and applying NBSE in feature space, we obtain a one-dimensional spectral embedding that reveals groups of redundant or semantically related dimensions; balanced binning then selects one representative per group. We prove that coloured Gaussian perturbations shift eta_N by at most O(arsigma^2), guaranteeing robustness to measurement noise. Experiments on ImageNet embeddings from MobileNetV2 and EfficientNet-B4 show that NBSE preserves classification accuracy even under aggressive compression: on EfficientNet-B4 the accuracy drop is below 1% when retaining only 30% of features, outperforming ANOVA F-test and random selection by up to 6.8%.

Related

• Scalable unsupervised feature selection via weight stability
• FSEVAL: Feature Selection Evaluation Toolbox and Dashboard
• Bi-Lipschitz Autoencoder With Injectivity Guarantee
• A Unified Fractional Regularization Framework for Sparse Recovery
• Assessing the impact of dimensionality reduction on clustering performance -- a systematic study

Source: arXiv cs.LG | 2026-04-28