Model Releases
Functional Degeneracy in Neural Networks: Measurement and Pruning
arXiv:2608.30741v1 Announce Type: new Abstract: A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the memory, compute
arXiv:2608.30741v1 Announce Type: new Abstract: A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the memory, compute and energy required to deploy it. To study this, we quantify functional degeneracy through the behavioral recovery rank, defined as the number of leading behavioral-Hessian eigendirections required to recover a trained model's performance. Using the behavioral recovery rank as a geometric benchmark for compression, we find that structural and magnitude pruning retain more degrees of freedom, even after the task is saturated. This gap suggests that functional redundancy is distributed across parameter directions and is not exposed by individual weights or neurons.
Related
- Fine-Tuning Integrity for Modern Neural Networks: Structured Drift Proofs via Norm, Rank, and Sparsity Certificates
- Natural Riemannian gradient for learning functional tensor networks
- Functional Similarity Metric for Neural Networks: Overcoming Parametric Ambiguity via Activation Region Analysis
Source: arXiv cs.LG | 2026-09-01