A Quotient Homology Theory of Representation in Neural Networks

arXiv:2502.01360v4 Announce Type: replace Abstract: Previous research has proven that the set of maps implemented by neural networks with a ReLU activation function is identical to the set of piecewise linear continuous maps. Furthermore, such networks induce a hyperplane arrangement splitting the input domain of the network into convex polyhedra G_J over which a network Phi operates in an affine manner. In this work, we leverage these properties to define an equivalence relation sim_Phi on top of an input dataset, which defines a quotient space that can be split into two sets related to the local rank of Phi_J and the intersections ap ext{Im}Phi_{J_i}. We refer to the latter as the extit{overlap decomposition} O_Phi and prove that if the intersections between each polyhedron and an input manifold are convex, the homology groups of neural representations are isomorphic to quotient homology groups H_k(Phi(M)) simeq H_k(M/O_Phi). This lets us intrinsically calculate the Betti numbers of neural representations without the choice of an external metric. We develop methods to numerically compute the overlap decomposition through linear programming and a union-find algorithm. Using this framework, we perform several experiments on toy datasets showing that, compared to standard persistent homology, our overlap homology-based computation of Betti numbers tracks purely topological rather than geometric features. Finally, we study the evolution of the overlap decomposition during training on several classification problems and discuss some shortcomings of our method.

Source: arXiv cs.LG | 2026-05-29