Research
State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking
arXiv:2608.03425v1 Announce Type: new Abstract: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms. However, for a class of d
arXiv:2608.03425v1 Announce Type: new Abstract: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms. However, for a class of deterministic state tracking tasks---such as parity checking, modular counting, and parenthesis matching---attention may be overkill. In this paper, we show that extbf{state propagation alone is sufficient}. We propose the extbf{Complex State Propagator (CSP)}, a minimalistic recurrent architecture that extbf{only propagates hidden states} across layers without output projections at intermediate steps. The state is represented as a complex-valued vector, updated via input-dependent rotations in the complex domain. To enable deep propagation without gradient vanishing or degradation, we introduce a extbf{block-level skip connection} alongside element-wise complex normalization and SiLU activation at sequence boundaries. Applied with Focal Loss, CSP achieves extbf{100% accuracy} with perfect F1 scores across canonical tasks.
Source: arXiv cs.AI | 2026-08-05