Beyond Affine Transformations: A Soft Dominance Layer for Coordinate-Wise Neural Computation

arXiv:2610.00563v1 Announce Type: cross Abstract: This paper presents a preliminary study of an alternative to the affine transformation underlying conventional neural-network layers. In the proposed Soft Dominance Layer, each output unit compares input coordinates with a learnable reference vector and aggregates smooth inequality responses. A sigmoid relaxation makes the comparisons differentiable, while a sharpness parameter alpha controls their transition toward hard threshold decisions. The aim is to examine the trainability and direct threshold interpretation of this primitive, not to claim a replacement for affine layers. In single-run MNIST experiments, the highest observed Soft Dominance accuracy is 0.9061 without annealing and 0.9173 with annealing, compared with 0.9827 for the MLP baseline. These descriptive results do not establish reliable configuration rankings or a statistically supported annealing benefit. Learned reference vectors exhibit spatial structure, providing qualitative evidence of structured learning. Repeated-seed experiments and broader datasets are required to assess robustness and practical relevance beyond this proof of concept.

Related

• LLMs Without Deep Neural Networks: New Architecture, Benefits and Case Study
• Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning
• Fractional Optimizers Meet Fractal Activation Functions: An Empirical Study of Multi-Scale Optimization in Neural Network

Source: arXiv cs.AI | 2026-10-02