Model Releases
A Mathematical Theory of Reusable Neural Bases for Network Compression
arXiv:2609.01550v1 Announce Type: cross Abstract: As large AI models become increasingly prevalent across a wide range of applications, memory cost has become a critical bottleneck in both training an
arXiv:2609.01550v1 Announce Type: cross Abstract: As large AI models become increasingly prevalent across a wide range of applications, memory cost has become a critical bottleneck in both training and inference. To mitigate this issue, we introduce the Linear Reusable Neural Bases Architecture (LRNBA), a novel framework aimed at improving parameter efficiency and reducing memory cost. Inspired by recurrent neural network (RNN) designs, the core idea of our approach is to represent each network block as a linear combination of a shared set of neural bases, thereby enjoying highly network compression rate while maintaining stable training. The proposed architecture allows for the construction of significantly wider and deeper networks under the same parameter budget. Extensive experiments demonstrate that our model achieves comparable or even faster convergence and lower loss than classical architectures, while maintaining stable training dynamics.
Related
- Break Through the Compression Bottleneck: From Theory to Practice
- OjaKV: Context-Aware Online Low-Rank KV Cache Compression
- QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy
- Efficient Pre-Training of LLMs through Truncated SVD Layers
- CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA
Source: arXiv cs.AI | 2026-09-02