A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities

arXiv:2607.19716v1 Announce Type: new Abstract: Pain is a complex and pervasive phenomenon affecting a large percentage of the population, and accurate assessment is essential for effective clinical management and intervention. Computational pain recognition systems enable continuous monitoring, support clinical decision-making, and help mitigate pain-related distress and functional decline. This study introduces a unified tokenization framework for heterogeneous 3D modalities in pain recognition that provides a single processing pipeline across behavioral and brain-activity 3D data, without requiring separate architectures for each modality or handcrafted inductive biases. The framework preserves spatial, temporal, and time--frequency structure while mapping diverse inputs into a shared token space. Extensive experiments show that the proposed approach effectively processes facial videos and fNIRS data in both raw-signal and spectrogram-based representations. On the AI4Pain benchmark dataset, the proposed framework achieves state-of-the-art performance while maintaining high computational efficiency and enabling real-time assessment on both GPU and CPU hardware.

Related

• Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods
• UNIV: Unified Foundation Model for Infrared and Visible Modalities
• A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of O(2)

Source: arXiv cs.CV | 2026-07-23