Model Releases
Understanding Clustering in Slot Attention via Particle Dynamics
arXiv:2610.04493v1 Announce Type: cross Abstract: Studying attention through the lens of interacting particle dynamics has shown how token clustering can emerge from the underlying dynamics. We extend
arXiv:2610.04493v1 Announce Type: cross Abstract: Studying attention through the lens of interacting particle dynamics has shown how token clustering can emerge from the underlying dynamics. We extend this perspective to slot attention, a method for object-centric image segmentation and representation learning in which learned components obscure how much of the clustering behaviour is intrinsic to the attention dynamics. We therefore introduce simplified slot attention (SSA), a parameter-free variant whose dynamics are connected to soft k-means clustering and which provides a straightforward mechanistic explanation for the emergence of object-centric representations. On the Pascal VOC dataset, SSA achieves performance comparable to that of slot attention, demonstrating that competitive object-centric segmentation can be achieved without learned neural-network components.
Related
- Spiking Local Interaction and Adaptive Complementary Fusion for Spiking Transformer
- WorldGraph: Graph-Native World Modeling
- QSV: Quat-Sphere-Vision for Coupled Quaternion Attention on Spherical Lattices
Source: arXiv cs.CV | 2026-10-06