Research
SO(3)-RoPE for Spherical Transformers
arXiv:2610.06229v1 Announce Type: new Abstract: Spherical data arise in many scientific applications. Often spherical transformers disregard the geometry of the underlying spherical domain, causing di
arXiv:2610.06229v1 Announce Type: new Abstract: Spherical data arise in many scientific applications. Often spherical transformers disregard the geometry of the underlying spherical domain, causing distortions and coordinate singularities near the poles. We introduce SO(3)-RoPE, a relative positional embedding that incorporates spherical geometry into transformer attention through unitary SO(3) representations. Our formulation is SO(3)-equivariant and compatible with FlashAttention, retaining efficiency of vanilla transformers. On shallow water dynamics prediction over a rotating sphere, our SO3ViT outperforms an S2Transformer baseline with lower errors and reduced runtime.
Related
- TAGTorch: A PyTorch Library for Geometry, Topology, and Symmetry-Aware Machine Learning
- LieDiscover: Adaptive Symbolic Library Construction for Explicit Open-form Symmetry Discovery
- Beyond Procrustes distances: a multilinear Gromov-Wasserstein distance capturing chirality
Source: arXiv cs.LG | 2026-10-06