CAGE-NAS: Certified Functional Descent for Efficient Model Growth

arXiv:2610.01173v1 Announce Type: new Abstract: The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be expanded. CAGE-NAS formulates this decision in function space through an admissibility criterion on approximations of the functional gradient. As long as a representation enables a certified Functional Gradient Descent step, the architecture remains fixed; when the criterion fails, a function-preserving expansion is applied and the resulting representation is evaluated again. As the main instance, we study the family induced by the tangent space, using a regularized projection of the functional gradient. In a controlled setting with exact certification, CAGE-NAS produces architectures positioned above the 99.8th performance percentile by held-out RMSE among all admissible alternatives within the same parameter budget, without enumerating them during the growth trajectory.

Related

• When Can Old Evaluations Certify a New Model? Label-Efficient Release Decisions under Evaluator Drift
• NObSP: Functional Decomposition of Neural Networks via Oblique Subspace Projections
• CacheDyG: Decoupling Temporal Propagation for Efficient Dynamic Graph Learning
• Efficient Dynamic Algorithms for Graph Neural Networks with Non-Linear Propagation

Source: arXiv cs.LG | 2026-10-02