Research
Emergent Self-Attention from Astrocyte-Gated Associative Memory Dynamics
arXiv:2604.25481v1 Announce Type: cross Abstract: We introduce a Hopfield-type associative memory in which effective connectivity is multiplicatively modulated by astrocytic gains evolving under an en
arXiv:2604.25481v1 Announce Type: cross Abstract: We introduce a Hopfield-type associative memory in which effective connectivity is multiplicatively modulated by astrocytic gains evolving under an entropy-regularized replicator equation. The coupled neuron-astrocyte dynamics admit a Lyapunov function, ensuring global convergence. At fixed points, astrocytic gains implement a softmax-normalized allocation over pattern similarity scores, yielding a mechanistic realization of self-attention as emergent routing on the gain simplex. In regimes of high memory load and interference, the model significantly improves retrieval accuracy relative to classical Hopfield dynamics and recent neuron-astrocyte baselines. These results establish a dynamical systems framework linking glial modulation, competitive resource allocation, and attention-like computation.
Related
- Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix
- Multi-layer Cross-Attention is Provably Optimal for Multi-modal In-context Learning
- MICA: Multivariate Infini Compressive Attention for Time Series Forecasting
- QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention
Source: arXiv cs.LG | 2026-04-29