Research
Understanding Deep Learning via Entropy Space Theory
arXiv:2608.29279v1 Announce Type: new Abstract: Deep learning is often criticized for its theoretical research lagging behind practice. To make deep learning easier to understand, the entropy space th
arXiv:2608.29279v1 Announce Type: new Abstract: Deep learning is often criticized for its theoretical research lagging behind practice. To make deep learning easier to understand, the entropy space theory is first introduced here. The entropy space can cover all the possibilities of any deep learning model by topological structure. It is independent of network parameters. Through the designed fundamental operations and norm, entropy space is proven to be a normed space within the formal axiomatic framework. Based on the theory, a unified coordinate system is proposed. It can coordinatize every state of a model and rank them by compression of the maximal value of information entropy. The theory offers a novel priori framework for mathematical fundamentals of deep learning.
Related
- Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless l^p Norm Solution for Fast Adversarial Training
- Understanding Deep Learning via Notions of Rank
- Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning
- Learning Tangent Bundles and Characteristic Classes with Autoencoder Atlases
Source: arXiv cs.AI | 2026-09-01