Model Releases
SignMatch: Matching Dictionary Signs to Continuous Sign Language Video
arXiv:2609.01886v1 Announce Type: new Abstract: The objective of this paper is to match dictionary sign videos to corresponding signs in continuous signing videos, where a match is defined by the visu
arXiv:2609.01886v1 Announce Type: new Abstract: The objective of this paper is to match dictionary sign videos to corresponding signs in continuous signing videos, where a match is defined by the visual similarity alone - the handshape and motion relative to the body. To achieve this, we learn a prototype-structured sign embedding space from continuous video annotated with signs, where each learnable prototype corresponds to a sign class. Isolated dictionary videos are then mapped into this sign space, enabling the matching between dictionary exemplars and continuous sign instances. This design supports direct dictionary-guided sign matching through embedding similarity and naturally extends to unseen signs using only dictionary exemplars. Experiments on ASL-Citizen dictionary retrieval, ChaLearn OSLWL dictionary-to-continuous sign matching, and using BOBSL's CSLR2 evaluation for automatic sign annotation demonstrate strong generalisation across datasets, tasks and sign languages. Without benchmark-specific supervision, the learned representation transfers effectively across American, British, and Spanish Sign Languages, outperforming prior methods on all three benchmarks. Project page: https://www.robots.ox.ac.uk/~vgg/research/signmatch/
Related
- JSL-DC: A Word-Level Japanese Sign Language Dataset with Linguist-Derived Descriptions for Distinguishing Confusable Signs
- SignNet-1M: Large-Scale Multilingual Sign Language Video Dataset with Downstream Benchmarks
- Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrieval
- Q-Fold: Query-Aware Focus-Context Spatio-Temporal Folding for Long Video Understanding
Source: arXiv cs.CV | 2026-09-03