S^3: Structured Sparsity Specification

arXiv:2604.11315v1 Announce Type: cross Abstract: We introduce the Structured Sparsity Specification (S^3), an algebraic framework for defining, composing, and implementing structured sparse patterns. S^3 specifies sparsity through three components: a View that reshapes the tensor via layout composition, a Block specification that defines the atomic pruning unit, and the sparsity decision Scope. Both Block and Scope support Coupling across tensors for coordinated sparsification. S^3 enables precise specification of diverse sparsity structures, from fine-grained N:M patterns to coarse channel pruning, and integrates seamlessly with Optimal Brain Damage (OBD) and Surgeon (OBS). We formalize the framework mathematically, demonstrate its expressiveness on canonical patterns, and validate it experimentally via structured OBS and OBD implementations built entirely on S^3, which surpasses well-established second order heuristics on output reconstruction across common configurations.

Related

• Dynamic sparsity in tree-structured feed-forward layers at scale
• NetworkNet: A Deep Neural Network Approach for Random Networks with Sparse Nodal Attributes and Complex Nodal Heterogeneity
• Beyond LLMs, Sparse Distributed Memory, and Neuromorphics <A Hyper-Dimensional SRAM-CAM 'VaCoAl' for Ultra-High Speed, Ultra-Low Power, and Low Cost>
• Toward a Tractability Frontier for Exact Relevance Certification

Source: arXiv cs.AI | 2026-04-14