Research
Generating Hadamard matrices with transformers
arXiv:2604.11101v1 Announce Type: cross Abstract: We present a new method for constructing Hadamard matrices that combines transformer neural networks with local search in the PatternBoost framework.
arXiv:2604.11101v1 Announce Type: cross Abstract: We present a new method for constructing Hadamard matrices that combines transformer neural networks with local search in the PatternBoost framework. Our approach is designed for extremely sparse combinatorial search problems and is particularly effective for Hadamard matrices of Goethals--Seidel type, where Fourier methods permit fast scoring and optimisation. For orders between 100 and 250, it produces large numbers of inequivalent Hadamard matrices, and in harder cases it succeeds where local search from random initialisation fails. The largest example found by our method has order 244. In addition to these new constructions, our experiments reveal that the transformer can discover and exploit useful hidden symmetry in the search space.
Related
- Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization
- Time-Frequency Analysis for Neural Networks
- Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks
- Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix
Source: arXiv cs.LG | 2026-04-14